TopolearnInput from users, files and APIs is regularly malformed or not what you expected.
json.loads and json.load raise json.JSONDecodeError on malformed input. It's a subclass of ValueError with useful attributes:
e.msgthe reason, e.g."Expecting value"or"Illegal trailing comma before end of object"(3.13 wording)e.lineno,e.colnoande.pos(character offset) for pointing at the probleme.docthe whole input
Common causes: an empty string (often an empty HTTP body), a trailing comma, single quotes, comments, or HTML from an error page where JSON was expected. Check the HTTP status and content type before parsing API responses.
Parsing successfully doesn't mean the data is correct. json.loads("42") is valid JSON but not the object you wanted. After decoding, check the shape:
if not isinstance(data, dict) or "id" not in data:
raise ValueError("expected an object with an id")For anything larger, define the expected structure once with a validation library such as Pydantic or jsonschema, or convert the dict into a dataclass in one place, so the rest of your code can trust the types.
Encoding can fail too: TypeError for unsupported types, and ValueError for NaN when allow_nan=False or for circular references.
Example
import json
def parse(text):
try:
data = json.loads(text)
except json.JSONDecodeError as e:
return f"line {e.lineno} col {e.colno}: {e.msg}"
if not isinstance(data, dict) or not isinstance(data.get("id"), int):
return "valid JSON, wrong shape"
return data
print(parse('{"id": 1}')) # {'id': 1}
print(parse('')) # line 1 col 1: Expecting value
print(parse('{"id": 1,}'))
# line 1 col 9: Illegal trailing comma before end of object
print(parse("{'id': 1}"))
# line 1 col 2: Expecting property name enclosed in double quotes
print(parse('[1, 2]')) # valid JSON, wrong shape
print(issubclass(json.JSONDecodeError, ValueError)) # TrueRemember
- Catch
json.JSONDecodeError(a ValueError) and reportlineno/colno - Empty strings and HTML error pages are the most common culprits
- Validate the decoded structure, not just the syntax
Watch out for
- Catching every exception around a whole request handler and hiding the decode position that would explain the bug.