TopolearnJSON encoding is often a hot spot in APIs and data pipelines.
The json module is implemented in Python with a C accelerator (_json) for scanning strings and encoding whole documents. Some calls bypass the fast paths:
json.dump(obj, f)streams throughiterencode, which uses the pure-Python encoder so it can yield chunks. For documents that fit in memory,f.write(json.dumps(obj))is often several times faster.- A
defaultfunction or encoder subclass costs one Python call per unsupported object. - Object hooks and
parse_float/parse_intcost one Python call per decoded object or number.
Measure with timeit on realistic payloads before optimising. Often the bigger costs are around JSON: building dicts from ORM objects, or validating with heavyweight schemas.
When JSON throughput really matters, popular third-party libraries are several times faster:
- orjson (Rust) serialises dataclasses, datetimes, UUIDs and NumPy arrays natively and returns
bytes. - msgspec decodes and validates directly into typed structs in one pass.
- ujson and simdjson bindings are other options.
They differ subtly from the stdlib: orjson returns bytes, rejects NaN, and limits integers to 64 bits; check these edges before swapping. For very large or columnar data, binary formats such as MessagePack, Protocol Buffers or Parquet beat any JSON library.
Example
import io
import json
import timeit
payload = {"items": [{"id": i, "name": f"item{i}", "price": i * 1.5,
"tags": ["a", "b"]} for i in range(1000)]}
one_shot = timeit.timeit(lambda: io.StringIO().write(json.dumps(payload)), number=50)
streamed = timeit.timeit(lambda: json.dump(payload, io.StringIO()), number=50)
print(streamed > one_shot) # True: dump streams via the Python encoder
text = json.dumps(payload)
plain = timeit.timeit(lambda: json.loads(text), number=50)
hooked = timeit.timeit(lambda: json.loads(text, object_hook=lambda d: d), number=50)
print(hooked > plain) # True: one Python call per object
from json import decoder
print(decoder.scanstring.__module__) # _json (C accelerator in use)Remember
- The stdlib json uses a C accelerator; streaming dump, hooks and default bypass it
- Measure realistic payloads before optimising
- orjson and msgspec are much faster but differ on edge cases
Watch out for
- Swapping in a faster library without checking how it handles NaN, big ints and return types.