Tool errors, caught and retried

A tool that raises stops the whole run. ToolErrorMiddleware turns the error into a message the model can read, and ToolRetryMiddleware calls the tool again first.

The shop's order database times out now and then. This version of the tool fails twice before it answers.

Exampletools.py

from langchain.tools import tool

OUTAGES = ["timed out", "timed out"]


@tool
def lookup_order(order_id: str) -> str:
    """Look up an order's shipping status by its id, such as A17."""
    if OUTAGES:
        raise ConnectionError(f"order database {OUTAGES.pop()}")
    return f"{order_id} shipped on 3 March."

Exampleagent.py

from langchain.agents import create_agent
from shop_model import ShopModel
from tools import lookup_order

Example

agent = create_agent(ShopModel(), tools=[lookup_order])
agent.invoke({"messages": [{"role": "user", "content": "Where is A17?"}]})

The exception came straight out of invoke. The model never heard about it, and the customer got no answer.

Errors as messages, and retries

Exampleagent.py, continued

from langchain.agents.middleware import ToolErrorMiddleware, ToolRetryMiddleware


def on_error(exc, request):
    return f"lookup failed: {exc}"


retry = ToolRetryMiddleware(max_retries=3, initial_delay=0, on_failure="error")
errors = ToolErrorMiddleware(on_error)

on_error receives the exception and the tool request, and returns the text of a tool message; returning None lets the exception through. ToolRetryMiddleware calls the tool again, here with no wait between attempts; its default is one second, doubling each time.

Example

agent = create_agent(ShopModel(), tools=[lookup_order], middleware=[retry, errors])
print(agent.invoke({"messages": [{"role": "user", "content": "Where is A17?"}]})["messages"][2].text)

This is the order the documentation's example uses, and the tool was never retried. From lesson 17, the first middleware is the outermost layer: errors sits inside retry, catches the first timeout and hands back a message, so retry sees a success.

Example

agent = create_agent(ShopModel(), tools=[lookup_order], middleware=[errors, retry])
print(agent.invoke({"messages": [{"role": "user", "content": "Where is A17?"}]})["messages"][2].text)

With errors first, retry is inside it and sees the raw exceptions. Two timeouts, a third attempt that worked, and a real answer. errors only gets involved if every attempt fails.

Try it yourself

Put three entries in OUTAGES and run the working order again.
Return None from on_error for ConnectionError and see the exception come back.
Print the status of the tool message when on_error handled the error.

PreviousChanging the call: wrap_model_call

This is what real progress feels like.