Parsing untrusted JSON safely

json · Expert · 11 min

Any public endpoint that calls json.loads on request bodies is exposed to these attacks.

json.loads is safe in the sense that it never executes code (unlike pickle or eval). But hostile input can still hurt:

  • Size: a 1 GB body becomes several GB of Python objects. Limit the request size before parsing, in your web server or framework.
  • Depth: [[[[...]]]] nested thousands of levels deep. The C decoder guards against stack overflow and raises RecursionError, which is not a ValueError, so a handler that catches only JSONDecodeError turns it into a 500 error. Catch it, or reject deeply nested input.
  • Huge integers: converting a number with hundreds of thousands of digits to int takes quadratic time. Since 3.11 (and security releases of older versions), CPython limits int/str conversion to 4300 digits by default and raises ValueError; sys.set_int_max_str_digits changes it. Pass parse_int to reject long numbers even earlier.
  • Duplicate keys: different parsers pick different values, which attackers use to slip data past a validator written in another language. Reject them with object_pairs_hook.
  • Type confusion: {"admin": "false"} is truthy; {"ids": {"$gt": ""}} sent to a database query. Validate types strictly after decoding.
  • NaN/Infinity: accepted by default, they poison comparisons and arithmetic. Reject with parse_constant.

Example

import json

def strict_loads(text, max_len=100_000):
    if len(text) > max_len:
        raise ValueError("body too large")
    def reject_constant(name):
        raise ValueError(f"{name} not allowed")
    def short_int(s):
        if len(s) > 30:
            raise ValueError("number too long")
        return int(s)
    try:
        return json.loads(text, parse_constant=reject_constant, parse_int=short_int)
    except RecursionError:
        raise ValueError("nesting too deep") from None

for body in ['{"n": 1}', '{"x": NaN}', "[" * 100_000 + "]" * 100_000,
             "9" * 50, "9" * 5000]:
    try:
        print(strict_loads(body, max_len=10**6))
    except ValueError as e:
        print("rejected:", str(e)[:24])

Remember

  • json never executes code, but size, depth and huge numbers can still DoS you
  • Deep nesting raises RecursionError, not JSONDecodeError
  • Reject NaN, long integers and duplicate keys with parse hooks

Watch out for

  • Catching only JSONDecodeError in an API handler; RecursionError and other ValueErrors escape as server errors.

Check yourself

  1. A request body is '[' * 200000 + ']' * 200000. What does json.loads raise in CPython 3.13?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/json.html#json.loads

Builds on

Unlocks

Nothing builds on this yet.

Parsing untrusted JSON safely · json · Topolearn