How botocore works: models and events

boto3 · Expert · 14 min

Understanding models and events lets you debug odd requests, add cross-cutting behavior and read any service's API precisely.

boto3 has almost no service-specific code. botocore ships a JSON service model per service and API version (botocore/data/<service>/<version>/service-2.json) describing operations, input and output shapes, errors and protocols; plus paginator and waiter definitions. When you call boto3.client("s3"), botocore loads the model and generates a client class at runtime, creating a method for each operation. That's why new AWS features appear by upgrading botocore, and why client creation is relatively expensive.

A call then flows through a pipeline:

  • Parameter validation against the input shape (ParamValidationError).
  • Serialization for the service's protocol (REST-JSON, JSON, query, REST-XML).
  • Endpoint resolution (region, FIPS, dual-stack, custom endpoints).
  • Signing with SigV4 using the resolved credentials.
  • Sending via urllib3, with retries handled by the retry handler.
  • Parsing the response into dicts (or raising modeled errors).

At each step, botocore emits events on the client's event system, such as provide-client-params.s3.PutObject, before-parameter-build.*, before-call.*, before-sign.*, after-call.* and needs-retry.*. Register handlers with client.meta.events.register("before-call.s3.PutObject", handler) to inject parameters, add headers, log or measure calls. Wildcards like before-call.*.* apply to all operations.

client.meta.service_model exposes the model at runtime: operation names, input shapes and documentation.

Example

import boto3

s3 = boto3.client("s3", region_name="us-east-1")
model = s3.meta.service_model
op = model.operation_model("PutObject")
print(model.protocol, op.http["method"], op.http["requestUri"])   # rest-xml PUT /{Bucket}/{Key+}
print(sorted(op.input_shape.required_members))                  # ['Bucket', 'Key']
print(len(model.operation_names) > 90)                          # True

def add_default_metadata(params, **kwargs):     # runs before parameters are built
    params.setdefault("Metadata", {})["app"] = "billing"

def log_call(model, params, **kwargs):
    print("calling", model.name, params["headers"].get("x-amz-meta-app"))

s3.meta.events.register("provide-client-params.s3.PutObject", add_default_metadata)
s3.meta.events.register("before-call.s3.PutObject", log_call)

s3.create_bucket(Bucket="demo-events-2026")
s3.put_object(Bucket="demo-events-2026", Key="k", Body=b"v")   # calling PutObject billing
print(s3.head_object(Bucket="demo-events-2026", Key="k")["Metadata"])   # {'app': 'billing'}

Remember

  • Clients are generated at runtime from JSON service models
  • Calls flow: validate, serialize, resolve endpoint, sign, send, parse
  • client.meta.events.register hooks into named lifecycle events
  • client.meta.service_model exposes operations and shapes

Watch out for

  • Registering a handler on the default session's events in library code, affecting every client in the process.

Check yourself

  1. Why does upgrading botocore add support for a new AWS API without boto3 code changes?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: boto3.amazonaws.com/v1/documentation/api/latest/guide/events.html