TopolearnProduction agents need context management, safety checks and cost limits without rewriting the agent loop.
Middleware plugs into the create_agent loop at defined points:
before_agent/after_agent: once per invocation.before_model/after_model: around every model call, e.g. trimming messages or checking outputs; they return state updates and can jump to the end.wrap_model_call(request, handler): wraps the call itself, so you can change the model, tools or system prompt per call, retry, or cache.wrap_tool_call(request, handler): wraps each tool execution for retries, logging or permission checks.
Write them as decorated functions (@before_model, @wrap_model_call, @dynamic_prompt) or subclasses of AgentMiddleware with state schemas and extra tools.
Built-in middleware covers common needs:
SummarizationMiddleware(model, trigger=("tokens", 4000), keep=("messages", 20))summarizes old history.ModelCallLimitMiddleware(run_limit=10)andToolCallLimitMiddlewarecap cost and loops.ModelFallbackMiddleware("openai:gpt-5-mini", "anthropic:claude-sonnet-4-5")switches models on errors;ModelRetryMiddleware,ToolRetryMiddlewareretry with backoff.PIIMiddlewareredacts or blocks sensitive data;HumanInTheLoopMiddlewarepauses for approval (next lesson).LLMToolSelectorMiddlewarenarrows large toolsets per query.
Middleware runs in list order for before_* hooks and in reverse for after_*; wrappers nest like decorators. Keep each one focused and test them with fake models.
Example
from langchain.agents import create_agent
from langchain.agents.middleware import (ModelCallLimitMiddleware, ModelRequest,
SummarizationMiddleware, dynamic_prompt)
from langchain_core.tools import tool
@tool
def lookup_order(order_id: str) -> str:
"""Look up an order's status."""
return f"Order {order_id}: shipped"
@dynamic_prompt
def tiered_prompt(request: ModelRequest) -> str:
tier = (request.runtime.context or {}).get("tier", "free")
return f"You are a support agent. Customer tier: {tier}. Be concise."
agent = create_agent(
model="openai:gpt-5-mini", tools=[lookup_order],
middleware=[tiered_prompt,
SummarizationMiddleware("openai:gpt-5-mini", trigger=("tokens", 4000)),
ModelCallLimitMiddleware(run_limit=8, exit_behavior="end")],
)
result = agent.invoke({"messages": [{"role": "user", "content": "Where is order 77?"}]},
context={"tier": "pro"})
print(result["messages"][-1].text) # Order 77 has shipped.Remember
- Hooks: before/after agent and model, plus wrap_model_call and wrap_tool_call
- Built-ins handle summarization, call limits, fallbacks, retries, PII and approvals
- Middleware composes in list order; keep each piece small and tested
Watch out for
- Implementing history trimming or retries inside every tool instead of once as middleware around the loop.