Testing code that uses Requests

Requests · Practical · 12 min

Tests that hit real APIs are slow, flaky and can't easily simulate errors; mocks make them fast and deterministic.

Code that calls an API should be tested without the API. Two common approaches:

The responses library (by Sentry, pip install responses) patches Requests' transport so registered URLs return canned responses. Inside @responses.activate (or the responses.RequestsMock() context manager), any request to an unregistered URL raises ConnectionError, so tests can never touch the network by accident. You can register JSON bodies, status codes, headers, exceptions (body=requests.ConnectionError()) and callbacks, and afterwards assert on responses.calls to check what your code sent.

Dependency injection is the other approach: design functions and clients to accept a Session (or a client object). Tests then pass a fake or a unittest.mock.Mock(spec=requests.Session). This keeps the code easy to test with any tool and makes it easy to share one session in production.

What to test:

  • the happy path parses data correctly,
  • error statuses (404, 500) and network errors lead to the behaviour you want,
  • the request is correct: URL, query parameters, headers, JSON body,
  • timeouts are passed (assert on the call arguments).

For integration tests against a real service, keep them separate and opt-in (for example a pytest marker), and use a local test server or recorded cassettes (vcrpy, pytest-recording) rather than production endpoints.

Example

import requests
import responses

def get_temperature(city, session=requests):
    r = session.get("https://api.example.com/weather", params={"city": city}, timeout=5)
    r.raise_for_status()
    return r.json()["temp"]

@responses.activate
def test_weather():
    responses.get("https://api.example.com/weather", json={"temp": 21.5})
    assert get_temperature("Yerevan") == 21.5
    assert responses.calls[0].request.params == {"city": "Yerevan"}

    responses.get("https://api.example.com/weather", status=503)
    try:
        get_temperature("Paris")
    except requests.HTTPError as e:
        print("handled", e.response.status_code)   # handled 503

test_weather()
print("ok")   # ok

Remember

  • The responses library intercepts Requests and blocks unregistered URLs
  • Accepting a session parameter makes code easy to fake
  • Test error paths and the exact request sent, not just success

Watch out for

  • Writing unit tests that call real third-party APIs.

Check yourself

  1. Inside @responses.activate, what happens when code requests a URL that was not registered?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: github.com/getsentry/responses