Stopping a runaway agent

A model can keep asking for tools and never answer. Call limits end the run after a set number of calls, so a stuck agent costs a few calls, not thousands.

Nothing in the agent loop counts. As long as the model keeps asking for tools, it keeps going. This model asks for the same lookup every time.

Examplestuck_model.py

from langchain.messages import AIMessage
from shop_model import ShopModel


class StuckModel(ShopModel):
    def decide(self, messages):
        call = {"name": "lookup_order", "args": {"order_id": "A17"}, "id": f"call_{len(messages)}"}
        return AIMessage("", tool_calls=[call])

Exampleagent.py

from langchain.agents import create_agent
from langchain.agents.middleware import ModelCallLimitMiddleware, ToolCallLimitMiddleware
from stuck_model import StuckModel
from tools import lookup_order

Example

agent = create_agent(StuckModel(), tools=[lookup_order])
agent.invoke({"messages": [{"role": "user", "content": "Where is A17?"}]}, {"recursion_limit": 10})

recursion_limit caps the number of steps, and here the run stopped with an error after ten. Left at the agent's default it goes on for 9,999 steps before this error. With a hosted model, that is thousands of paid calls for one question.

A limit on model calls

Example

limit = ModelCallLimitMiddleware(run_limit=3)
agent = create_agent(StuckModel(), tools=[lookup_order], middleware=[limit])
result = agent.invoke({"messages": [{"role": "user", "content": "Where is A17?"}]})

print(len(result["messages"]))
print(result["messages"][-1].text)

Three model calls, then the run ended with an AI message saying why. run_limit counts calls in one invoke; thread_limit counts across a whole thread when there is a checkpointer. The default exit_behavior of "end" finishes with that message; "error" raises instead.

A limit on one tool

Example

limits = [ToolCallLimitMiddleware(tool_name="lookup_order", run_limit=2),
          ModelCallLimitMiddleware(run_limit=4)]
agent = create_agent(StuckModel(), tools=[lookup_order], middleware=limits)

for message in agent.invoke({"messages": [{"role": "user", "content": "Where is A17?"}]})["messages"]:
    if message.type == "tool":
        print(message.text)

Two lookups ran. After that the tool call limit answered each request itself, telling the model to stop, and the model call limit ended the run. A tool limit's default behaviour is "continue": block the tool, keep the agent going.

Try it yourself

Set exit_behavior="error" on the model call limit and read the exception.
Give the tool limit no tool_name, so it counts every tool.
Run lesson 7's agent with a model call limit of 1 and ask about A17.

PreviousTool errors, caught and retried

Every expert started right here.