Tasks: running coroutines concurrently

asyncio · Essentials · 10 min

Awaiting coroutines one by one runs them sequentially; Tasks are how you actually get concurrency.

await coro() runs a coroutine and waits for it right there. Two awaits in a row run one after the other. To run things at the same time, schedule them as Tasks.

task = asyncio.create_task(coro()) wraps the coroutine in a Task and schedules it on the event loop. It starts running at the next await point of the current coroutine. You can then do other work and later await task to get its result (or its exception).

A Task is a Future subclass with useful methods:

  • task.done(), task.result(), task.exception(),
  • task.cancel() to request cancellation,
  • task.add_done_callback(fn), task.get_name().

One rule matters a lot: keep a reference to tasks you create. The event loop only holds weak references, so a fire-and-forget task with no other reference can be garbage-collected mid-run. Store tasks in a variable or a set, or better, use a TaskGroup, which owns its tasks for you.

Also make sure every task is eventually awaited. If a task fails and nobody awaits it, the exception is only logged as "Task exception was never retrieved" when the task is destroyed, which is easy to miss.

Example

import asyncio, time

async def fetch(name, delay):
    await asyncio.sleep(delay)
    return f"{name} done"

async def main():
    t0 = time.perf_counter()
    a = asyncio.create_task(fetch("a", 1.0))
    b = asyncio.create_task(fetch("b", 1.0))
    print(a.done())                       # False: scheduled, not finished
    print(await a, await b)               # a done b done
    print(f"{time.perf_counter() - t0:.1f}s")  # 1.0s, not 2.0s

asyncio.run(main())

Remember

  • create_task schedules a coroutine to run concurrently
  • Await the task later to get its result or exception
  • Keep a reference to every task you create

Watch out for

  • Creating a task and dropping the reference; it may be garbage-collected or its error silently lost.

Check yourself

  1. Why does a program that awaits fetch('a') then fetch('b') take twice as long as one using create_task?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/asyncio-task.html#creating-tasks