Middleware: code around the model

Middleware is code that runs at fixed points in the agent loop, such as before every model call. It can read the agent's state, change it, or end the run.

The agent from lesson 7 is a loop: call the model, run the tools it asks for, repeat. Middleware attaches to the points in that loop. Four hooks run at those points: before_agent and after_agent once per invoke, and before_model and after_model around every model call.

from langchain.agents.middleware import after_model, before_model


@before_model
def count(state, runtime):
    print("before the model:", len(state["messages"]), "messages")


@after_model
def report(state, runtime):
    last = state["messages"][-1]
    print("after the model: ", last.text or last.tool_calls[0]["name"])

A decorated function becomes middleware. It receives the current state, with its messages, and the runtime from lesson 9. Returning nothing leaves the state as it was.

Hooks in the loop

from langchain.agents import create_agent
from hooks import count, report
from shop_model import ShopModel
from tools import lookup_order

agent = create_agent(ShopModel(), tools=[lookup_order], middleware=[count, report])

agent.invoke({"messages": [{"role": "user", "content": "Where is A17?"}]})

Two model calls, so each hook ran twice. The first time the model saw the question and asked for lookup_order. The second time it saw three messages, the question, its request and the tool result, and answered.

Middleware is part of the loop

for step in agent.stream({"messages": [{"role": "user", "content": "Where is A17?"}]}, stream_mode="updates"):
    print("step:", list(step))

Streaming the same agent shows the hooks as steps of their own, named after the function and the hook. They run inside the agent's loop, so everything built on top of it, including memory and human approval, works with them.