Testing agents with pytest and TestModel

Tests should never call a paid model. agent.override swaps in TestModel or your own function, and ALLOW_MODEL_REQUESTS=False fails any test that forgets to.

The code under test, support.py, uses a real model name, and an answer function that the rest of the app calls:

Examplesupport.py

from pydantic_ai import Agent

ORDERS = {"A-1001": "shipped on 12 March"}

agent = Agent("openai:gpt-5.2", defer_model_check=True, instructions="You answer support tickets.")


@agent.tool_plain
def lookup_order(order_id: str) -> str:
    """Look up the status of an order."""
    return ORDERS.get(order_id, "not found")


def answer(ticket: str) -> str:
    return agent.run_sync(ticket).output

Blocking real requests

Example

models.ALLOW_MODEL_REQUESTS = False
os.environ["OPENAI_API_KEY"] = "sk-test"

answer("Where is A-1001?")

With ALLOW_MODEL_REQUESTS set to False, a real model refuses to send anything, even with a key present. Put it at the top of your test files, so a test that is not using a stand-in fails loudly instead of spending money.

override

Example

with agent.override(model=TestModel()):
    with capture_run_messages() as messages:
        print(answer("Where is A-1001?"))

for message in messages:
    print(message.kind, [part.part_kind for part in message.parts])

agent.override(model=...) replaces the model for every run inside the with, even deep inside answer, which your test cannot pass a model to. It also takes deps=, for the dependencies from lesson 11. capture_run_messages() collects the messages of a run started inside it, so a test can check which tools were called.

TestModel is good for checking that the wiring works: every tool is called and its schema accepted. Its arguments are made up, so it cannot test what your tools do with real values. A model function can:

Example

pytest -q

Exampletest_support.py

from pydantic_ai import capture_run_messages, models
from pydantic_ai.models.test import TestModel

from support import agent, answer
from shop_model import shop_model

models.ALLOW_MODEL_REQUESTS = False


def test_answer_runs_with_the_tool():
    with agent.override(model=TestModel()):
        with capture_run_messages() as messages:
            answer("Where is A-1001?")
    call = messages[1].parts[0]
    assert call.tool_name == "lookup_order"


def test_known_order_is_found():
    with agent.override(model=shop_model):
        assert answer("Where is A-1001?") == "Order A-1001: shipped on 12 March."


def test_unknown_order():
    with agent.override(model=shop_model):
        assert "not found" in answer("Where is A-4040?")

Three tests, no key, no network. The shop model gives A-1001 and an unknown order to the tool, so the tests check your tool's real behaviour. Pydantic AI also hides its banner when it runs under pytest.

Try it yourself

Add a test that passes TestModel(custom_output_text="Hello") and checks answer returns it.
Remove the override from one test and run pytest again.
Write a pytest fixture that wraps a test in agent.override(model=shop_model).

PreviousAgents that call agents

Every expert started right here.