Output validators: rules Pydantic cannot check

Some rules need your data, like whether an order exists. An output validator runs after Pydantic and can send the model back with ModelRetry.

A ticket should carry an order id only if that order is real. Pydantic can check that order_id is a string, but not that it is in your database. The model function here copies whatever looks like an id, and drops it after a retry:

import re
from typing import Literal

from pydantic import BaseModel
from pydantic_ai import Agent, ModelResponse, ModelRetry, RunContext, ToolCallPart
from pydantic_ai.models.function import FunctionModel

ORDERS = {"A-1001": "shipped", "A-1002": "waiting for stock"}


class Ticket(BaseModel):
    category: Literal["billing", "shipping", "other"]
    order_id: str | None


def reader(messages, info):
    last = messages[-1].parts[-1]
    ticket = messages[0].parts[-1].content
    found = re.search(r"A-\d+", ticket)
    order_id = None if last.part_kind == "retry-prompt" or not found else found.group()
    return ModelResponse(parts=[ToolCallPart("final_result", {"category": "billing", "order_id": order_id})])

agent = Agent(FunctionModel(reader), output_type=Ticket)


@agent.output_validator
def order_exists(ctx: RunContext, ticket: Ticket) -> Ticket:
    print("attempt", ctx.retry)
    if ticket.order_id is not None and ticket.order_id not in ORDERS:
        raise ModelRetry(f"There is no order {ticket.order_id}. Use null if the ticket has no valid order id.")
    return ticket

print(agent.run_sync("I was charged twice for order A-1001").output)
print("---")
print(agent.run_sync("I was charged twice for order A-10001").output)

@agent.output_validator registers a function that gets the validated Ticket. Returning it accepts it. Raising ModelRetry sends your message to the model as a retry prompt, the same as a validation error in lesson 7, and uses the same retry allowance.

For A-1001 it passed on attempt 0. The typo A-10001 is not an order, so the model was asked again, and on attempt 1 it returned order_id=None. The validator's first argument, a RunContext, describes the run, and ctx.retry counts the retries so far. Lesson 11 puts your own data in it.

Validator or Pydantic?

• A rule about the value itself, like a range or a pattern, belongs on the Pydantic model with Field or a field_validator. It also shows up in the schema the model reads.
• A rule that needs a database, an API or the run's context belongs in an output validator. It can be async.

Write the ModelRetry message for the model: what was wrong and what to do instead. It is the only thing the model learns about the failure.