TopolearnA description nobody tests drifts from reality, and every tool built on it inherits the lies.
The contract is only useful if the running API honours it. Layers of defence:
Schema-driven fuzzing. Schemathesis reads the document and generates many requests (valid, edge-case and invalid) with property-based testing, then checks responses: status codes documented, bodies conform to schemas, no 5xx errors, content types right. It finds undocumented error responses and validation gaps quickly. Run it in CI against a test deployment.
Runtime validation. Middleware validates real requests (and optionally responses) against the document: express-openapi-validator for Express, openapi-core for Python, or built-in validation in code-first frameworks. Validating responses in staging catches drift as soon as it appears.
Proxy validation. Prism's proxy mode or API gateways check live traffic against the contract without code changes.
Consumer-driven contracts. Pact-style tests capture what each client actually uses; with bi-directional contract testing, consumer pacts are compared against the provider's OpenAPI document, so you know whether a change breaks a real consumer, not just a theoretical one.
Make the document honest first: realistic examples, complete error responses, additionalProperties: false where appropriate, and accurate required lists. Then test-driven drift checks become cheap and reliable.
Example
# Property-based testing of every operation
pip install schemathesis
st run ./openapi.yaml --url http://localhost:8000 \
--checks all --max-examples 50 \
-H "Authorization: Bearer $TEST_TOKEN"
# FAILED: POST /books
# Undocumented HTTP status code: 500
# Reproduce with: curl -X POST http://localhost:8000/books -d '{"title": ""}' ...
# Runtime validation middleware in an Express app:
# import * as OpenApiValidator from "express-openapi-validator";
# app.use(OpenApiValidator.middleware({
# apiSpec: "./openapi.yaml",
# validateRequests: true,
# validateResponses: process.env.NODE_ENV !== "production",
# }));Remember
- Fuzz the API from its schema with Schemathesis
- Validate requests (and responses in staging) with middleware
- Pair the provider contract with consumer-driven tests
- Complete error responses and examples make tests meaningful
Watch out for
- Generating the document from code and assuming it is therefore correct; annotations drift too.