Real models: providers, keys and model names

A real model is named as a provider and a model id, like openai:gpt-4.1, and the provider's client reads its API key from an environment variable.

Last updated: 28 Sep, 2026 · Pydantic AI 2.51

The stand-in from FunctionModel: write a stand-in model needed no key. To send a ticket to a hosted model you name it as a single string and set the matching key. This is the one line that changes when you leave the stand-in.

Naming a provider and reading its key

With no OPENAI_API_KEY set, creating the agent fails at once, and the message says what to set:

from pydantic_ai import Agent

agent = Agent("openai:gpt-4.1", instructions="You answer support tickets.")

Traceback (most recent call last):
  File "main.py", line 3, in <module>
    agent = Agent("openai:gpt-4.1", instructions="You answer support tickets.")
pydantic_ai.exceptions.UserError: Set the `OPENAI_API_KEY` environment variable or pass it via `OpenAIProvider(api_key=...)` to use the OpenAI provider. To try Pydantic AI without an API key, use the built-in test model: `Agent('test')`. See https://pydantic.dev/docs/ai/guides/testing/

With the key exported, the same line works and run_sync sends the ticket to the provider:

export OPENAI_API_KEY="sk-..."

import re

from pydantic_ai import ModelResponse, TextPart, ToolCallPart
from pydantic_ai.models.function import AgentInfo, FunctionModel


def sort_ticket(text):
    text = text.lower()
    if "charged" in text or "refund" in text:
        return "billing", 4
    if "parcel" in text or "arrived" in text:
        return "shipping", 3
    return "other", 1


def shop_reply(messages, info: AgentInfo) -> ModelResponse:
    prompts = [p.content for m in messages for p in m.parts if p.part_kind == "user-prompt"]
    ticket = prompts[-1]
    last = messages[-1].parts[-1]
    order = re.search(r"A-\d{4}", ticket)

    # 1. The ticket names an order and the agent has a tool: ask for it.
    if order and info.function_tools and last.part_kind == "user-prompt":
        tool = info.function_tools[0].name
        return ModelResponse(parts=[ToolCallPart(tool, {"order_id": order.group()})])

    # 2. A tool answered: write the reply from what it said.
    if last.part_kind == "tool-return" and info.allow_text_output:
        return ModelResponse(parts=[TextPart(f"Order {order.group()}: {last.content}.")])

    # 3. The agent wants a typed answer: fill in its output tool.
    category, priority = sort_ticket(ticket)
    if info.output_tools:
        args = {"category": category, "priority": priority}
        return ModelResponse(parts=[ToolCallPart(info.output_tools[0].name, args)])

    # 4. Otherwise, plain text.
    return ModelResponse(parts=[TextPart(f"Sorted as {category}.")])


shop_model = FunctionModel(shop_reply, model_name="shop")

Choosing the model when you run

An agent does not need a model when it is created. Passing model= on a run picks one for that run and overrides the agent's own model. Here the stand-in fills in:

agent = Agent(instructions="You answer support tickets.")
print(agent.run_sync("My parcel never arrived", model=shop_model).output)

Sorted as shipping.

With no model on the agent and none on the run, the run fails:

agent = Agent(instructions="You answer support tickets.")
agent.run_sync("My parcel never arrived")

Traceback (most recent call last):
  File "main.py", line 2, in <module>
    agent.run_sync("My parcel never arrived")
pydantic_ai.exceptions.UserError: `model` must either be set on the agent or included when calling it.

Creating an agent before the key exists

agent = Agent("openai:gpt-4.1", defer_model_check=True)
print(agent.run_sync("I was charged twice", model=shop_model).output)

Sorted as billing.

defer_model_check=True waits until the first run that uses the named model before building it. Your module can then be imported on a machine without keys, such as a test runner, as long as those runs use another model. The testing and capstone parts rely on this.

Reaching many providers through the Gateway

The Pydantic AI Gateway sits between your agent and the providers. One Gateway key reaches several providers, and the model string gains a gateway/ prefix. The rest of the agent stays the same:

from pydantic_ai import Agent

agent = Agent("gateway/openai:gpt-4.1", instructions="You answer support tickets.")

export PYDANTIC_AI_GATEWAY_API_KEY="pylf_v..."

What the Gateway adds is around the calls: spending limits per project or key, failing over between providers that serve the same model, and every request logged in Pydantic Logfire. Turn it on from Logfire and create a key there.

Running a model on your own machine

Ollama runs open models locally and speaks the OpenAI API, so Pydantic AI connects with the OpenAI model class and an Ollama provider:

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.ollama import OllamaProvider

model = OpenAIChatModel("qwen2.5:3b", provider=OllamaProvider(base_url="http://localhost:11434/v1"))
agent = Agent(model, instructions="You answer support tickets.")

Small local models are weaker at calling tools and filling typed output than hosted ones, so expect more retries, covered in Validation retries: when the model gets it wrong, if you try the course with one.

Related

• Previous: Instructions: what the model is told
• Next: Structured output: a typed ticket
• Reference: Models overview