TopolearnFanning out many requests and collecting the answers is the bread-and-butter asyncio pattern.
await asyncio.gather(a(), b(), c()) schedules each awaitable as a task, waits for all of them, and returns a list of results in the order you passed them, not the order they finished. Pass a generated list with a star: await asyncio.gather(*(fetch(u) for u in urls)).
Error handling is the part to understand:
- By default, the first exception raised by any awaitable propagates out of
gather. The other tasks are not cancelled; they keep running in the background. - With
return_exceptions=True, exceptions are returned in the result list in place of values, so you can inspect every outcome.
If you want "fail fast and cancel the rest", use asyncio.TaskGroup (Python 3.11+), which is now the recommended way to run a group of related tasks. gather remains handy for quick fan-outs, for collecting per-item failures with return_exceptions=True, and in code that must support older versions.
When fanning out to hundreds of items, limit concurrency (for example with a Semaphore) so you don't open 10,000 connections at once.
Example
import asyncio
async def square(n):
await asyncio.sleep(0.1 * (5 - n)) # later items finish first
if n == 3:
raise ValueError("bad 3")
return n * n
async def main():
ok = await asyncio.gather(square(1), square(2))
print(ok) # [1, 4] (argument order)
res = await asyncio.gather(*(square(n) for n in range(1, 5)),
return_exceptions=True)
print(res) # [1, 4, ValueError('bad 3'), 16]
try:
await asyncio.gather(square(3), square(4))
except ValueError as e:
print("raised:", e) # raised: bad 3
asyncio.run(main())Remember
- gather returns results in argument order
- The first error propagates but sibling tasks keep running
- return_exceptions=True puts exceptions into the results list
Watch out for
- Assuming gather cancels the other tasks when one fails; it doesn't.