TopolearnConcurrency is the main reason to choose HTTPX: hundreds of requests overlap instead of running one by one.
httpx.AsyncClient has the same API as Client, but its methods are coroutines: await client.get(url). Use it inside async code with async with:
async with httpx.AsyncClient(base_url=...) as client:
r = await client.get("/items")The payoff is concurrency. While one request waits for the network, the event loop runs others. To fetch many URLs, create the coroutines and run them together with asyncio.gather (or a TaskGroup on Python 3.11+). Total time is roughly the slowest request rather than the sum.
Unbounded concurrency hurts both you and the server, so limit it:
- an
asyncio.Semaphore(n)around each request caps in-flight requests, - the client's
limits=httpx.Limits(max_connections=...)caps connections; extra requests wait for a free one (and may hit the pool timeout).
Rules for async code:
- Share one
AsyncClientacross tasks; do not create one per request. - Never call the sync
httpx.getinside async code: it blocks the event loop. - Async event hooks and auth must be async too.
AsyncClientworks with asyncio and Trio (through AnyIO).
In FastAPI or other async frameworks, create the client at start-up (lifespan) and close it at shutdown with await client.aclose().
Example
import asyncio
import time
import httpx
async def fetch(client, sem, n):
async with sem: # at most 5 requests in flight
r = await client.get(f"/delay/1", params={"n": n})
return r.json()["args"]["n"]
async def main():
sem = asyncio.Semaphore(5)
async with httpx.AsyncClient(base_url="https://httpbin.org", timeout=10) as client:
t = time.perf_counter()
results = await asyncio.gather(*(fetch(client, sem, i) for i in range(10)))
print(results) # ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']
print(f"{time.perf_counter() - t:.0f}s") # 2s, not 10s
asyncio.run(main())Remember
- AsyncClient mirrors Client with awaitable methods
- asyncio.gather or TaskGroup runs requests concurrently
- Bound concurrency with a semaphore and share one client
Watch out for
- Calling httpx.get (sync) inside an async function, which blocks the event loop.