TopolearnDownloads larger than memory, progress bars and server-sent events all need streaming.
Normally HTTPX reads the whole response body into memory. For large downloads or long-lived streams, use client.stream(method, url) as a context manager:
with client.stream("GET", url) as r:
for chunk in r.iter_bytes():
...Inside the block the headers are available immediately, and you choose how to consume the body:
iter_bytes()decoded bytes (gzip and other content encodings removed),iter_text()decoded text,iter_lines()line by line (great for NDJSON and server-sent events),iter_raw()the raw bytes exactly as sent over the wire, still compressed.
For async clients use async with client.stream(...) and the aiter_bytes(), aiter_text(), aiter_lines(), aiter_raw() versions.
The connection is released when the block exits, even if you stop reading early. Outside the block the body is gone: accessing r.content on a streamed response that you did not read raises httpx.ResponseNotRead. If you decide mid-way that you want the whole body, call r.read() (or await r.aread()).
r.num_bytes_downloaded tracks progress through the raw stream, which, combined with Content-Length, drives a progress bar.
Request bodies can be streamed too: pass a generator of bytes (or an async generator for AsyncClient) as content=. HTTPX sends it with chunked transfer encoding, so the data never has to be in memory at once. Some servers reject chunked uploads; if you know the size in advance, set a Content-Length header and HTTPX sends the stream without chunking.
Example
import httpx
with httpx.Client(base_url="https://httpbin.org") as client:
with client.stream("GET", "/bytes/50000") as r:
total = int(r.headers["Content-Length"])
received = 0
for chunk in r.iter_bytes(chunk_size=8192):
received += len(chunk)
print(received == total, r.num_bytes_downloaded) # True 50000
with client.stream("GET", "/stream/3") as r: # newline-delimited JSON
print(sum(1 for line in r.iter_lines() if line)) # 3
def body():
for part in (b"chunk1,", b"chunk2"):
yield part
r = client.post("/post", content=body(), headers={"Content-Length": "13"})
print(r.json()["data"]) # chunk1,chunk2Remember
- client.stream(...) as r plus iter_bytes/iter_lines consumes the body incrementally
- Async variants are aiter_bytes, aiter_lines, ...
- Generators passed as content= stream the request body
Watch out for
- Accessing r.content outside the stream block, which raises ResponseNotRead.