Scopes and Checkpoints

Part 4 took you from yield to Task. The protocol that came out of it — a coroutine yielding a Future, a Task pumping it with send, cancellation as a throw into the paused yield — is the floor everything in this post sits on. anyio doesn't replace any of it. It bolts a second discipline on top: lexical cancellation, structured spawning, and an explicit word for the place those two meet.

Three primitives, all of them ordinary Python objects you can pass around, store, and compose. CancelScope. TaskGroup. MemoryObjectStream. The whole library has more, but these are the load-bearing ones — the ones the next post weaponizes.

§1Two models, side by side

asyncio's cancellation is per-task: task.cancel() arranges for CancelledError to be thrown into task's coroutine at its next resume. You hold a Task object; you cancel a Task object. The unit of cancellation is the running thing.

anyio's cancellation is per-scope: scope.cancel() trips the scope, and every task that is currently executing inside that lexical region gets unwound at its next checkpoint. You hold a CancelScope; you cancel a region. The unit of cancellation is the bracket in source.

Same machinery — coro.throw(CancelledError()) under the hood. Different shape of the API. The shape matters because it scales differently. Cancelling a single task is the easy case. Cancelling "everything below this point in the call tree" is the case real programs need, and asyncio makes you write it by hand.

§2CancelScope: a cancellable lexical region

A CancelScope is a context manager. The block under it is the region. You can cancel the scope at any time — before entering it, inside it, from outside. Timeouts are a scope with a deadline.

import anyio

async def demo():
    with anyio.CancelScope() as scope:
        await anyio.sleep(0.1)
        scope.cancel()                  # trip it from inside
        await anyio.sleep(10)              # this raises CancelledError
        print("never reached")

    print("after the scope")         # reached — the scope swallows its OWN cancellation

with anyio.move_on_after(5):           # scope with a deadline
    await something_slow()
                                       # after 5s the scope cancels itself, control resumes here

Three properties worth memorising. First, a scope swallows the CancelledError that it itself raised — so control resumes after the with block, not by propagating the exception upward. This is what makes timeouts feel ergonomic: you don't write a try/except at every call site. Second, a scope does not swallow cancellations from outer scopes — those re-raise so the outer scope can do its own cleanup. Third, you can cancel a scope before anyone enters it, and the first checkpoint inside will fire immediately. That last property is what the cancellation post will weaponize next.

§3What a checkpoint actually checks

"Cancellation lands at the next checkpoint." Fine — but what is a checkpoint, mechanically? anyio's own definition, real source:

# anyio/lowlevel.py
async def checkpoint() -> None:
    """Check for cancellation and allow the scheduler to switch to another task."""
    await get_async_backend().checkpoint()

A checkpoint is exactly two things welded together: a chance for the scheduler to run someone else, and a look at every CancelScope this task is lexically inside to see if any of them is now cancelled. Every await on something that actually parks (Part 4, §6 — a Future the loop hasn't resolved) is a checkpoint. Awaiting a coroutine that runs straight through without parking is not. That distinction was the whole trap in §3 of the old draft, and it is load-bearing for the next post.

A checkpoint is the moment the task suspends and the moment cancellation checks fire — bolted together so you can't have one without the other.

Common Python operations that hit a checkpoint: await anyio.sleep(...), awaiting a stream read, awaiting a lock acquisition, awaiting a queue get when the queue is empty. Operations that don't necessarily suspend: tg.start_soon, entering a task group, dictionary lookups, arithmetic. If you want a guaranteed scheduler-yielding moment, call await anyio.lowlevel.checkpoint() explicitly.

§4How cancellation actually lands

The mechanism is sneakier than "throw at next yield". When you call scope.cancel(), anyio does not throw anything right now. It schedules a callback that will throw later. Real source, trimmed to the load-bearing lines:

# anyio/_backends/_asyncio.py — CancelScope._deliver_cancellation
def _deliver_cancellation(self, origin):
    current = current_task()
    for task in self._tasks:
        if task is not current and ...:   # never cancels the running task inline
            waiter = task._fut_waiter
            if not isinstance(waiter, asyncio.Future) or not waiter.done():
                task.cancel(origin._cancel_reason)
    ...
    if origin is self and should_retry:
        self._cancel_handle = get_running_loop().call_soon(
            self._deliver_cancellation, origin)         # try again next loop tick

Two highlighted lines: skip the currently-running task; if work remains, reschedule yourself.

Read it slowly. _deliver_cancellation walks every task registered with this scope. For each one that is not the currently running task, it calls task.cancel() — which under the hood schedules a CancelledError to be thrown into that task at its next resume. The currently-running task is skipped on purpose: cancelling it inline would mean raising CancelledError right here inside cancel(), which the caller would have to be ready for. Instead, anyio reschedules _deliver_cancellation via loop.call_soon — to run again on the next loop tick.

So calling scope.cancel() does not raise. It arms a callback. The cancellation actually lands at the next moment the task suspends — when _fut_waiter is set to a Future the task is parked on. Then the next loop tick fires, _deliver_cancellation runs again, finds the task parked, calls task.cancel() on it, the asyncio machinery throws CancelledError at the paused yield self, and the exception starts unwinding.

§5Task groups: structured concurrency

asyncio lets you fire-and-forget: asyncio.create_task(coro()) spawns a task, returns immediately, and the task lives until the loop ends or someone cancels it. If you forget to await it, exceptions get swallowed; if you forget to cancel it, it leaks. Half the bugs in the SSE field report are forgotten tasks.

anyio inherits trio's answer: TaskGroup. You can only spawn tasks inside one, and the group is a context manager that cannot exit until every task it spawned has finished. The block is a hard wall around the children.

async def worker(n):
    await anyio.sleep(n)
    print(f"worker {n} done")

async def main():
    async with anyio.create_task_group() as tg:
        tg.start_soon(worker, 1)
        tg.start_soon(worker, 2)
        print("spawned")
    print("all workers finished")         # reached only AFTER both workers return

Properties this gets you. If worker(1) raises, the group cancels every other child and re-raises the exception at the async with exit. No silently-swallowed errors. If the parent is cancelled (because some outer scope tripped), the group propagates the cancellation to every child, waits for them to unwind, and only then re-raises. No leaked tasks. The promise — no task outlives the block that spawned it — is the whole point of structured concurrency.

The promise has a cost, paid on exit. That's the next section.

§6Free to enter, expensive to exit

Entering a task group does not suspend. Read the source:

# anyio/_backends/_asyncio.py — TaskGroup.__aenter__
async def __aenter__(self) -> TaskGroup:
    self.cancel_scope.__enter__()         # synchronous! no await inside
    self._active = True
    return self

The body awaits nothing. cancel_scope.__enter__() is an ordinary synchronous method — it appends the scope to the task's lexical-scope stack and returns. So await tg.__aenter__() runs start to finish without ever yielding a Future to the loop. Part 4, §5 told you this is allowed: async grants the right to suspend; it doesn't force one. Entering a task group is, for scheduling purposes, a function call with extra syntax — not a checkpoint.

Exiting is the opposite. The promise — no task outlives the block — means __aexit__ cannot return until every child has finished. So __aexit__ creates a Future and awaits it, parking the host task until the last child completes. Real source:

# anyio/_backends/_asyncio.py — TaskGroup.__aexit__ (trimmed)
if self._tasks:
    with CancelScope() as wait_scope:
        while self._tasks:
            self._on_completed_fut = loop.create_future()
            await self._on_completed_fut     # suspends until a child completes

That await self._on_completed_fut is a real, full suspension — a checkpoint. It is the place where any pre-armed cancellation will finally fire. The task group's cost is paid at the join, not at the spawn. Free to enter, expensive to exit.

§7Memory streams have a back wall

One more primitive. asyncio gives you Queue(maxsize=…). anyio gives you the memory object stream: a sender half and a receiver half, with a buffer between. The shape that matters is send:

# anyio/streams/memory.py — MemoryObjectSendStream (trimmed)
def send_nowait(self, item):
    ...
    if len(self._state.buffer) < self._state.max_buffer_size:
        self._state.buffer.append(item)       # room → returns immediately
    else:
        raise WouldBlock                         # full → caller must await send()

With max_buffer_size=0 (the default) the buffer has no room ever: every send blocks until a receiver is waiting to take the item directly. That's a rendezvous — pure backpressure, the sender can never outrun the receiver. With a positive size, the sender may run ahead by that many items before it's forced to wait. Either way there is a back wall: the producer cannot push unboundedly.

This is the property a plain unbounded asyncio.Queue lacks, and it's the reason Starlette routes streaming-response bytes through a memory stream — to couple the producer's pace to the client's. The failure mode in the field report is what happens when that coupling sits behind a layer that has stopped reading: the sender parks on the back wall, the layer above can't peek through it, and the streaming response stalls. Part 6 takes that apart.

§8Take with you

• asyncio cancels a Task; anyio cancels a scope. Same throw mechanism underneath; different unit. Lexical cancellation composes when per-task cancellation doesn't.
• A checkpoint is a real suspension and a cancellation check, welded. Awaiting something that parks is a checkpoint; awaiting a coroutine that runs straight through is not.
• scope.cancel() does not raise. It arms a call_soon callback. The cancellation lands at the next checkpoint. No suspension, no delivery.
• A task group is a hard wall. No task outlives the block. The exit awaits the join — and that await is the checkpoint a pre-armed scope will fire on.
• A memory stream has a back wall. Buffer size is the slack between producer and consumer; default zero means rendezvous. The producer cannot outrun the consumer.