Sessions: connection reuse and defaults

Requests · Practical · 10 min

Sessions make repeated calls to one API faster and keep login state, and they are how real clients are structured.

Each top-level requests.get creates a throwaway session and a new connection. A requests.Session keeps state across requests:

  • Connection pooling. Requests to the same host reuse open TCP and TLS connections (HTTP keep-alive). For many calls to one API this is often a large speedup, because the TLS handshake is skipped.
  • Cookies. Cookies set by responses are stored in s.cookies and sent automatically on later requests, so logins work like in a browser.
  • Defaults. Attributes like s.headers, s.params, s.auth, s.verify and s.proxies apply to every request. Per-request arguments are merged with them; for dicts, a per-request key overrides the session key, and setting a key to None removes it for that request.

Use a session as a context manager so its connections are closed when you are done:

with requests.Session() as s:
    s.headers["Authorization"] = f"Bearer {token}"
    s.get(...)

In a larger application, create one session per API client (for example in a small wrapper class with a base_url) and reuse it, instead of creating a session for each request.

One gotcha: there is no session-wide default timeout attribute. Setting s.timeout = 5 does nothing. Pass timeout= on each call, or wrap the session (subclass and override request, or use a helper function) to apply a default.

Example

import requests

with requests.Session() as s:
    s.headers.update({"User-Agent": "demo/1.0", "Accept": "application/json"})
    s.params = {"api_key": "abc"}

    s.get("https://httpbin.org/cookies/set/session_id/42", timeout=10)
    print(s.cookies.get("session_id"))          # 42

    r = s.get("https://httpbin.org/get", params={"page": 1}, timeout=10)
    echo = r.json()
    print(echo["args"])        # {'api_key': 'abc', 'page': '1'}  (merged)
    print(echo["headers"]["User-Agent"])        # demo/1.0

    r = s.get("https://httpbin.org/headers", headers={"Accept": None}, timeout=10)
    print("Accept" in r.json()["headers"])      # False: removed for this request

    r = s.get("https://httpbin.org/cookies", timeout=10)
    print(r.json())                              # {'cookies': {'session_id': '42'}}

Remember

  • A Session reuses connections and persists cookies
  • Session headers, params, auth and verify are merged into each request
  • There is no session-level timeout; pass timeout= per request

Watch out for

  • Creating a new Session for every request, which throws away connection reuse.

Check yourself

  1. You set s.timeout = 5 on a Session. What timeout do requests made with s.get(url) use?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: requests.readthedocs.io/en/latest/user/advanced/#session-objects