FallbackModel: when a provider is down

Providers have outages. FallbackModel tries a list of models in order and moves to the next when one fails with an API error.

from pydantic_ai import Agent
from pydantic_ai.exceptions import ModelHTTPError
from pydantic_ai.models.fallback import FallbackModel
from pydantic_ai.models.function import FunctionModel

from shop_model import shop_model


def overloaded(messages, info):
    raise ModelHTTPError(status_code=503, model_name="primary", body="overloaded")


primary = FunctionModel(overloaded, model_name="primary")

overloaded behaves like a provider returning HTTP 503, by raising the same ModelHTTPError Pydantic AI raises for a real one.

agent = Agent(FallbackModel(primary, shop_model))
result = agent.run_sync("My parcel never arrived")
print(result.output)
print(result.response.model_name)

FallbackModel(first, second, ...) is itself a model. primary failed, so the same request went to shop_model, and result.response.model_name says which model answered. In production the list would be real models from different providers, such as "openai:gpt-5.2" then "anthropic:claude-sonnet-4-6".

def backup_down(messages, info):
    raise ModelHTTPError(status_code=500, model_name="backup", body="internal error")


agent = Agent(FallbackModel(primary, FunctionModel(backup_down)))
try:
    agent.run_sync("My parcel never arrived")
except FallbackExceptionGroup as group:
    print(group)
    for error in group.exceptions:
        print("  ", error)

When every model fails, you get a FallbackExceptionGroup holding each error.

What does not fall back

By default only ModelAPIError, the errors from calling a provider, moves to the next model. A validation error, lesson 7, or ModelRetry from a tool is a retry with the same model; a model that answers badly is not down. fallback_on= takes other exception types if you need them.