Structured Concurrency & Cancellation Propagation Playbook

Date: 2026-02-28
Category: knowledge
Domain: software / distributed systems / reliability engineering

Why this matters

Most async production bugs are not “algorithm bugs.” They are lifecycle bugs:

• orphaned tasks still running after request timeout,
• partial failures that keep expensive sibling work alive,
• hidden goroutine/thread/task leaks,
• missing deadline propagation across service boundaries.

Structured concurrency fixes this by making concurrency obey a parent-child tree:

• children cannot outlive parent scope,
• failure and cancellation follow explicit rules,
• scope exit means “all subtasks accounted for.”

That turns async code from “best effort cleanup” into an enforceable contract.

Core mental model (portable across languages)

A concurrent request should look like this:

1. Parent scope starts with deadline + cancellation token.
2. Parent forks child tasks for independent subtasks.
3. Policy decides behavior on first child failure:
   • fail-fast: cancel siblings and return error,
   • supervised: isolate child failure, continue others.
4. Parent joins children before returning.
5. Scope exit guarantees no child is left running.

If your runtime lets children escape that lifecycle, you’re back in unstructured land.

Policy choices you must make explicitly

1) Fail-fast scope (default for request handlers)

Use when subtasks are semantically one unit (e.g., aggregate profile + entitlements + risk).

• first non-retryable failure cancels siblings,
• saves budget and latency tail,
• keeps error semantics simple.

2) Supervisor scope (for partial-value workflows)

Use when child outputs are optional/independent.

• one child can fail without killing others,
• caller must own fallback semantics and degradation policy,
• requires clearer result typing (success | partial | failed).

Rule: if you don’t have a principled partial-response policy, pick fail-fast.

Language mapping (practical semantics)

Java (StructuredTaskScope)

• Structured concurrency API models subtasks as one unit.
• join() enforces synchronization at scope boundary.
• ShutdownOnFailure and ShutdownOnSuccess provide short-circuit policies.
• Scope shutdown cancels unfinished subtasks (interrupt-based cancellation model).

Operational note: treat interruption as a first-class cancellation signal in blocking code.

Kotlin Coroutines

• coroutineScope {}: child failure fails scope and cancels siblings.
• supervisorScope {}: child failure does not cancel siblings by default.
• Parent cancellation propagates downward through Job hierarchy.

Operational note: choose coroutineScope vs supervisorScope deliberately per call path; do not mix by habit.

Go (context + errgroup.WithContext)

• context.Context forms a cancellation/deadline tree.
• errgroup.WithContext cancels sibling goroutines on first error.
• Wait() is the join point for task completion and error propagation.

Operational note: every blocking select/send/recv/path should listen to ctx.Done().

Python (asyncio.TaskGroup)

• TaskGroup gives structured lifetime for sibling tasks.
• Exiting the context joins children; failure handling is coordinated by the group.
• Cancellation is central; swallowing CancelledError can break structured behavior.

Operational note: cancellation should usually be re-raised after cleanup.

Implementation blueprint for service handlers

Use this sequence in API orchestration paths:

1. Ingress

   • derive request deadline from SLO budget,
   • attach trace/span + request id.

2. Scope create

   • create one structured scope per user request,
   • pick policy: fail-fast or supervisor.

3. Fan-out

   • fork child calls with inherited deadline/context,
   • isolate each child with local timeout < parent budget.

4. Cancellation wiring

   • child libraries must honor cancellation quickly,
   • retries/backoff must stop when parent cancels.

5. Join + classify

   • wait at scope boundary,
   • classify result as ok, partial, failed, canceled, deadline_exceeded.

6. Cleanup & observability

   • emit cancellation cause + winning error,
   • confirm zero leaked workers/tasks after request completion.

Failure modes seen in production

1. Fire-and-forget inside request scope

   • creates detached work and post-timeout side effects.

2. Timeout without propagation

   • edge times out, downstream keeps working (resource burn).

3. Retry loop ignoring cancellation

   • service keeps retrying after client already left.

4. Swallowing cancellation exceptions

   • task appears “successful” while caller was canceled.

5. Supervisor semantics used accidentally

   • silent partial responses with no product contract.

6. No bounded concurrency at child layer

   • fan-out spike creates self-inflicted overload.

Metrics that reveal lifecycle health

Track these per endpoint:

• request_canceled_total (by cause: client, timeout, parent failure)
• child_task_canceled_total
• inflight_child_tasks (gauge)
• orphan_task_detected_total (from leak tests / runtime guards)
• scope_join_latency_ms (time between first failure and full teardown)
• deadline_budget_remaining_ms at downstream call start

Healthy systems show fast cancellation convergence and near-zero orphan counts.

Testing checklist (must-pass before production)

1. Sibling cancellation test

   • one child fails immediately; others must exit within bounded time.

2. Parent timeout test

   • parent deadline fires; all children stop and release resources.

3. Slow child test

   • one child hangs; ensure watchdog + cancellation prevents tail hostage.

4. Retry cancellation test

   • cancel during backoff; retry loop must stop instantly.

5. Leak test

   • run N repeated requests and assert no growth in live tasks/goroutines/threads.

6. Partial response contract test (if supervisor mode)

   • verify degraded payload is explicit and user-safe.

Decision quick guide

• Request aggregation endpoint? → Fail-fast structured scope.
• Background batch with independent units? → Supervisor + bounded worker pool.
• Unsure? → Start fail-fast, add supervised islands only with explicit product semantics.

One-page policy template

• Every async request path must have one root cancellation/deadline context.
• No detached child tasks inside request path unless explicitly transferred to durable job system.
• Every child task must be joinable or explicitly persisted.
• Cancellation handling is part of API contract and code review checklist.
• CI includes leak + cancellation-convergence tests.

References (researched)

• OpenJDK JEP 505: Structured Concurrency (Fifth Preview)
  https://openjdk.org/jeps/505
• Oracle Java docs: Structured Concurrency / StructuredTaskScope
  https://docs.oracle.com/en/java/javase/21/core/structured-concurrency.html
• Kotlin Coroutines API: coroutineScope
  https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/coroutine-scope.html
• Kotlin Coroutines API: supervisorScope
  https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/supervisor-scope.html
• Go blog: Context (cancellation/deadline tree model)
  https://go.dev/blog/context
• Go package docs: golang.org/x/sync/errgroup
  https://pkg.go.dev/golang.org/x/sync/errgroup
• Python docs: asyncio Coroutines and Tasks (TaskGroup, cancellation)
  https://docs.python.org/3/library/asyncio-task.html