Usage limits: stopping a run that loops

A model can keep calling tools without ever answering. Usage limits cap requests, tool calls and tokens, and end the run when one is reached.

from pydantic_ai import Agent, ModelResponse, ToolCallPart, UsageLimits
from pydantic_ai.models.function import FunctionModel


def impatient(messages, info):
    return ModelResponse(parts=[ToolCallPart("check_refund", {"order_id": "A-1001"})])


agent = Agent(FunctionModel(impatient))
checks = 0

@agent.tool_plain
def check_refund(order_id: str) -> str:
    """Check whether a refund has been paid."""
    global checks
    checks += 1
    return "still pending"

This model asks whether a refund has been paid, hears "still pending", and asks again, forever.

try:
    agent.run_sync("Has my refund been paid?")
except UsageLimitExceeded as error:
    print(error)
print("checks:", checks)

A run has a limit even when you set none: 50 requests. It stopped the loop after 50 requests and 50 tool calls. At a real model's prices and speed, that is already far too much for one ticket.

try:
    agent.run_sync("Has my refund been paid?", usage_limits=UsageLimits(request_limit=4))
except UsageLimitExceeded as error:
    print(error)
print("checks:", checks)

UsageLimits, passed to a run, sets the caps. The request limit is checked before each request, so the one that would go over is never sent.

Token limits are checked against the counts a response reports, so the request that goes over has already been paid for. count_tokens_before_request=True counts input tokens first, with an extra call to the provider. A limit per ticket is a budget: pick it from what a normal ticket uses, which result.usage tells you.