Cancellation and cleanup

asyncio · Practical · 12 min

Timeouts, TaskGroups and server shutdowns all work through cancellation, so your code must cooperate with it.

task.cancel() requests cancellation: the next time the task is suspended at an await, a CancelledError is thrown into it. The exception unwinds the coroutine, running finally blocks and async with exits on the way, so resources get released.

Rules for cancellation-safe code:

  • Put cleanup in finally or a context manager. Cleanup may itself await (closing a connection, for example).
  • If you catch CancelledError to log or tidy up, re-raise it. Swallowing it breaks asyncio.timeout(), TaskGroup and asyncio.run() shutdown, which all rely on the cancellation completing.
  • CancelledError subclasses BaseException, so except Exception does not catch it. That's deliberate.

After a task is cancelled, task.cancelled() is True and awaiting it raises CancelledError. cancel("reason") attaches a message.

asyncio.shield(aw) protects an inner awaitable from cancellation of the outer one. The outer await still raises CancelledError, but the shielded operation keeps running, which is useful for work that must not be interrupted halfway, such as committing a transaction. Hold a reference to the shielded task so it isn't lost.

Cancellation only lands at awaits; a CPU loop with no awaits cannot be cancelled until it reaches one.

Example

import asyncio

async def worker():
    try:
        while True:
            await asyncio.sleep(0.1)
    except asyncio.CancelledError:
        print("worker: cancelled, closing")
        raise                              # always re-raise
    finally:
        print("worker: finally ran")

async def main():
    t = asyncio.create_task(worker())
    await asyncio.sleep(0.25)
    t.cancel("shutting down")
    try:
        await t
    except asyncio.CancelledError:
        print("main: cancelled() =", t.cancelled())

asyncio.run(main())
# worker: cancelled, closing
# worker: finally ran
# main: cancelled() = True

Remember

  • cancel() throws CancelledError into the task at its next await
  • Clean up in finally / async with; re-raise CancelledError if you catch it
  • shield() protects an inner operation from outer cancellation

Watch out for

  • Wrapping a loop body in except BaseException (or except CancelledError) without re-raising, making the task uncancellable.

Check yourself

  1. Why does except Exception not catch asyncio.CancelledError?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/asyncio-task.html#task-cancellation