LangChain (YT style)LangChain 1.4 · Python 3.12+
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
46 small wins to finish your pathNext lesson →

Validation errors, and the retry

Validation is the check a schema runs on the model's answer, and by default create_agent sends any error back to the model as a tool message so it can try again.

Last updated: 27 Sep, 2026 · LangChain 1.4

A written stand-in model here
This lesson runs on TicketModel, a small stand-in model written below, not on Groq. The point is what happens when a model sends a value the schema rejects, and a real model will not send a bad value on cue. The stand-in always makes the same mistake, so the error and the retry show up every run.

A free-text status is hard to count, so the ticket system accepts three words and the schema says so with Literal. A model that copies the lookup result, "waiting for stock", straight into the status no longer fits.

The ToolStrategy wrapper

python
from langchain.agents.structured_output import ToolStrategy

strategy = ToolStrategy(Ticket, handle_errors=True)   # True (default) sends errors back to the model
agent = create_agent(model, tools=[...], response_format=strategy)

Tightening the schema with Literal

Tighten the schema so the status must be one of three words.

Example
from typing import Literal

from pydantic import BaseModel


class Ticket(BaseModel):
    """A support ticket for one order."""
    order_id: str
    status: Literal["shipped", "waiting", "unknown"]

A stand-in that copies the lookup text

TicketModel is a chat model built like the ShopModel in the BaseChatModel lesson. It follows a fixed script: look the order up, then copy the lookup text into Ticket. Start with the imports, its type name and bind_tools, which create_agent calls.

python
import re

from langchain.chat_models import BaseChatModel
from langchain.messages import AIMessage
from langchain_core.outputs import ChatGeneration, ChatResult


class TicketModel(BaseChatModel):
    @property
    def _llm_type(self):
        return "ticket"

    def bind_tools(self, tools, **kwargs):
        return self                     # its script already knows the tool names

_generate picks the next tool call from the last message. After the customer's question it asks for lookup_order; after the lookup result it calls Ticket with that text as the status; after an error it calls fix, shown in the retry section.

python
    def _generate(self, messages, stop=None, run_manager=None, **kwargs):
        last = messages[-1]
        if last.type == "human":                    # 1. look the order up
            order_id = re.findall(r"\b[A-Z]\d+\b", last.text)[0]
            call = {"name": "lookup_order", "args": {"order_id": order_id}, "id": "call_lookup"}
        elif last.text.startswith("Error"):         # 3. the retry, further down
            call = self.fix(messages)
        else:                                       # 2. copy the lookup text into Ticket
            order_id, status = last.text.rstrip(".").split(" ", 1)
            call = {"name": "Ticket", "args": {"order_id": order_id, "status": status}, "id": "call_ticket"}
        return ChatResult(generations=[ChatGeneration(message=AIMessage("", tool_calls=[call]))])

fix pulls the allowed words out of the error message, picks the one that appears in the lookup result, and calls Ticket again. A real model does this by reading the error itself; this method is only how the stand-in imitates that.

python
    def fix(self, messages):
        error = messages[-1].text
        allowed = re.findall(r"'(\w+)'", error.split("Input should be")[1].split("[")[0])
        found = next(m.text for m in messages if m.type == "tool" and m.name != "Ticket")
        status = next((word for word in allowed if word in found), "unknown")
        return {"name": "Ticket", "args": {"order_id": found.split(" ")[0], "status": status}, "id": "call_retry"}

The lookup tool

This lesson's agent answers order questions with lookup_order, the tool built in Tools: a function the model can call. Add it below the models.

python
from langchain.tools import tool

ORDERS = {"A17": "shipped on 3 March", "C40": "waiting for stock"}


@tool
def lookup_order(order_id: str) -> str:
    """Look up an order's shipping status by its id, such as A17."""
    status = ORDERS.get(order_id)
    return f"{order_id} {status}." if status else f"{order_id} is not an order we have."

The raw error, with handling off

To see the raw error first, turn error handling off with handle_errors=False. Asking about C40 makes TicketModel send "waiting for stock" as the status, which the Literal rejects.

Example
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy

strict = ToolStrategy(Ticket, handle_errors=False)
agent = create_agent(TicketModel(), tools=[lookup_order], response_format=strict)

agent.invoke({"messages": [{"role": "user", "content": "Where is C40?"}]})

With handle_errors=False the Pydantic error ends the run. It names the field, the value the model sent, and the three that were allowed.

Reading the error and trying again

handle_errors defaults to True: the error goes back to the model as a tool message, and the loop continues. A real model reads it and corrects its call; TicketModel does the same through its fix method. Leave it on its default and the same question recovers. Print the message types, the tool call in each AI message, and the error sent back in between.

Example
from langchain.agents import create_agent
from langchain.agents.structured_output import ToolStrategy

agent = create_agent(TicketModel(), tools=[lookup_order], response_format=ToolStrategy(Ticket))
result = agent.invoke({"messages": [{"role": "user", "content": "Where is C40?"}]})

print(repr(result["structured_response"]))
print([m.type for m in result["messages"]])
for message in result["messages"]:
    if message.type == "ai":
        print(message.tool_calls[0]["name"], message.tool_calls[0]["args"])
error = next(m for m in result["messages"] if m.type == "tool" and m.text.startswith("Error:"))
print(error.text)

How the retry recovered

  • The message types show the whole run: the question, the lookup and its result, the first Ticket call and the error sent back for it, then the corrected Ticket call and the tool message that ended the loop.
  • The first Ticket call sent "waiting for stock"; the retry sent waiting, the allowed word found inside it, and that is the ticket that came back.
  • The error, three messages from the end, names the field, the bad value and the three allowed words, and ends by asking the model to fix its mistake.

handle_errors False vs True

handle_errorsWhat happens on a bad value
FalseThe validation error is raised and the run stops
True (default)The error goes back to the model as a tool message and the loop continues

When to handle validation errors

  • Letting a model recover from a value your schema rejects without crashing the run.
  • Failing fast in tests, where you want a bad value to raise instead of retry.
Watch out. handle_errors=True only helps if the model reads the returned error and changes its next call. A model that sends the same value again loops until it hits the call limit, so keep the schema's message clear about what is allowed.
Try it yourself
  • Ask about A17 and check which of the three words its status becomes.
  • Print result["messages"][-2].tool_calls and find the id of the corrected call.
  • Remove "waiting" from the Literal and see which word fix chooses for C40.
PreviousStructured output with response_format

You understood something today that you didn't yesterday.