TLS verification, certificates and proxies

Requests · Advanced · 11 min

Corporate networks, internal CAs and mutual TLS are everyday realities, and disabling verification is the wrong fix.

Requests verifies HTTPS certificates by default against the CA bundle from the certifi package. A failed check raises requests.exceptions.SSLError.

When a server uses an internal or corporate CA, do not turn verification off. Point Requests at the right bundle instead:

  • verify="/path/to/ca-bundle.pem" per request or on a session,
  • or the REQUESTS_CA_BUNDLE environment variable (also CURL_CA_BUNDLE).

verify=False disables all checks, which makes man-in-the-middle attacks trivial; urllib3 emits an InsecureRequestWarning for each such request. Keep it for throwaway local experiments.

For mutual TLS, where the server authenticates the client, pass cert=("client.crt", "client.key") or a single PEM file containing both.

Proxies are configured with proxies={"http": "...", "https": "..."}, keyed by the scheme of the target URL. Proxy URLs can contain credentials (http://user:pass@proxy:3128), and SOCKS proxies work with pip install requests[socks] and a socks5h:// URL (the h resolves DNS through the proxy).

Requests also honours the environment: HTTP_PROXY, HTTPS_PROXY, NO_PROXY, REQUESTS_CA_BUNDLE and .netrc credentials. This is controlled by session.trust_env (default True). Setting it to False makes behaviour independent of the environment, which can help in tests but surprises users who rely on corporate proxy variables. Note that environment settings take precedence over session attributes like s.verify in some cases, while per-request arguments always win.

Example

import requests

s = requests.Session()
print(s.verify, s.trust_env)            # True True

# Internal CA: point at the bundle, don't disable verification
# s.verify = "/etc/ssl/certs/corp-ca.pem"

# Mutual TLS: client certificate and key
# s.cert = ("client.crt", "client.key")

s.proxies = {"https": "http://proxy.internal:3128"}
print(s.proxies["https"])               # http://proxy.internal:3128

settings = s.merge_environment_settings(
    "https://example.com", proxies={}, stream=None, verify=None, cert=None)
print(sorted(settings))                 # ['cert', 'proxies', 'stream', 'verify']

try:
    requests.get("https://self-signed.badssl.com/", timeout=10)
except requests.exceptions.SSLError:
    print("certificate rejected")        # certificate rejected
except requests.ConnectionError:
    print("no network")

Remember

  • Verification is on by default; fix CA problems with verify=path or REQUESTS_CA_BUNDLE
  • cert= supplies a client certificate for mutual TLS
  • proxies= and the HTTP(S)_PROXY / NO_PROXY variables route traffic; trust_env controls env use

Watch out for

  • Setting verify=False to silence an SSLError in production code.

Check yourself

  1. Your company uses an internal CA and requests raises SSLError. What is the right fix?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification