Prepared requests and the request lifecycle

Requests · Expert · 13 min

Debugging odd API behaviour and building advanced clients requires knowing how arguments become bytes on the wire.

requests.get(...) is a thin wrapper. Internally:

  • A Request object is created from your arguments: method, URL, headers, params, data, json, files, auth, cookies, hooks.
  • Session.prepare_request merges it with session defaults (headers, cookies, auth, params) and calls PreparedRequest.prepare, which builds the final URL with encoded query string, the body bytes, the Content-Type and Content-Length headers, and applies auth (including .netrc).
  • Session.send merges environment settings (proxies, verify, cert from the environment when trust_env is true), picks the transport adapter whose mounted prefix best matches the URL, and calls adapter.send.
  • The adapter turns the prepared request into a urllib3 call, builds a Response, and Session.send runs response hooks, extracts cookies, and follows redirects by building new prepared requests (resolve_redirects), which also handles method changes on 303 and auth stripping.

You can drive this manually: req = requests.Request("POST", url, json=...), prep = s.prepare_request(req), adjust prep.headers or prep.body, then s.send(prep, timeout=...). This is useful for signing the exact bytes being sent, for logging, and for sending something the high-level API would normalise.

Note send does not apply merge_environment_settings for you; call it yourself if you need proxies and CA settings from the environment. Also, prep.body is bytes or a string after preparation, not your dict.

Example

import hashlib
import requests

with requests.Session() as s:
    s.headers["User-Agent"] = "lifecycle-demo"
    req = requests.Request("POST", "https://httpbin.org/post",
                           params={"v": 2}, json={"id": 7})
    prep = s.prepare_request(req)          # merges session defaults

    print(prep.url)                        # https://httpbin.org/post?v=2
    print(prep.body)                       # b'{"id": 7}'
    print(prep.headers["Content-Type"])    # application/json
    print(prep.headers["User-Agent"])      # lifecycle-demo

    prep.headers["X-Body-SHA256"] = hashlib.sha256(prep.body).hexdigest()[:12]
    settings = s.merge_environment_settings(prep.url, {}, None, None, None)
    r = s.send(prep, timeout=10, **settings)
    print(r.json()["headers"]["X-Body-Sha256"])  # ad559f4e2220

Remember

  • Request -> prepare_request (merge defaults) -> PreparedRequest -> adapter.send
  • Prepared requests expose the final URL, headers and body bytes
  • Session.send skips environment merging; call merge_environment_settings yourself

Watch out for

  • Expecting s.send(prep) to pick up proxy and CA settings from the environment automatically.

Check yourself

  1. Which object holds the final encoded body bytes that go on the wire?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: requests.readthedocs.io/en/latest/user/advanced/#prepared-requests