LiteLLMLiteLLM 1.105 · Python 3.10+
Dashboard
0%

Structured output your code can read

Structured output is a request option that makes a model reply with JSON that fits a schema you give it, so your code can read the reply as data instead of parsing prose.

Last updated: 12 Oct, 2026 · LiteLLM 1.105

Tool calling: when the model asks for data had the model send arguments as JSON. This lesson asks for the whole reply in a shape you choose, such as a support ticket with a category and an urgency flag. LiteLLM takes the schema in one form and passes it to each provider in the form that provider expects.

Syntax: pass a Pydantic class (or a JSON schema dictionary) as response_format, then validate the reply text with the same class.

python
response = completion(
    model="groq/openai/gpt-oss-20b",
    messages=messages,
    response_format=Ticket,                 # the shape the reply must have
    max_tokens=300,
)
ticket = Ticket.model_validate_json(response.choices[0].message.content)

Describing the reply with a Pydantic class

The schema as a class

Each attribute becomes a field the reply must contain, and its type becomes the type of the value. LiteLLM turns the class into a JSON schema before sending it.

python
from pydantic import BaseModel

class Ticket(BaseModel):
    category: str      # billing, shipping or other
    urgent: bool

Checking that a model supports schemas

Not every model accepts a schema. litellm.supports_response_schema() answers from LiteLLM's model table, without a call.

ExampleAll three course models accept a schema
import litellm

for model in ["groq/openai/gpt-oss-120b", "groq/openai/gpt-oss-20b", "gemini/gemini-2.5-flash-lite"]:
    print(model, litellm.supports_response_schema(model))

Reading the reply as an object

The reply text is JSON. model_validate_json parses it and checks every field in one step, and it returns a Ticket, not a dictionary.

ExampleAPI keyThe same schema on two providers
from pydantic import BaseModel

class Ticket(BaseModel):
    category: str      # billing, shipping or other
    urgent: bool

from litellm import completion

messages = [{"role": "user", "content":
             "Classify this message. Category is billing, shipping or other: "
             "My card was charged twice, fix it today!"}]
for model in ["groq/openai/gpt-oss-20b", "gemini/gemini-2.5-flash-lite"]:
    response = completion(model=model, messages=messages, response_format=Ticket, max_tokens=300)
    text = response.choices[0].message.content
    print(repr(text))
    print(Ticket.model_validate_json(text))
  • The text differs, the data does not. The text can differ in spacing: in this run Gemini spread the JSON over several lines and gpt-oss returned one compact line. Both parse to category='billing' urgent=True.
  • Validate, do not compare strings. Spacing and key order are the provider's choice, so code that matches the text exactly breaks on a provider change.

What happens without a schema

Before schemas, the usual request was {"type": "json_object"}, which asks for valid JSON of any shape. Groq adds a rule to it: the word json must appear somewhere in the messages.

Examplejson_object without the word json in the prompt
import litellm
from litellm import completion

litellm.suppress_debug_info = True     # hides LiteLLM's help banner on errors

try:
    completion(
        model="groq/openai/gpt-oss-20b",
        messages=[{"role": "user", "content": "Classify: My card was charged twice."}],
        response_format={"type": "json_object"},
        max_tokens=300,
    )
except litellm.BadRequestError as error:
    print(type(error).__name__)
    print(error)

LiteLLM raised BadRequestError and carried Groq's message in the error text. The schema form in the earlier examples needed no such word. Even with the word added, json_object promises valid JSON, not your fields.

Catching a reply that does not fit

A reply that is valid JSON but missing a field passes json.loads and fails later, at the line that needs the field. Validating with the class fails at once and names the field.

ExampleA reply with no urgent field
from pydantic import BaseModel

class Ticket(BaseModel):
    category: str      # billing, shipping or other
    urgent: bool

from pydantic import ValidationError

try:
    Ticket.model_validate_json('{"category": "billing"}')
except ValidationError as error:
    print([(item["loc"], item["type"]) for item in error.errors()])

The error names the missing field, urgent, and its type, missing. A program can catch it and ask the model again.

json_object vs a schema

json_objectA schema (Pydantic class)
You send{"type": "json_object"}The class or a JSON schema dictionary
Reply is valid JSONAsked forAsked for
Reply has your fieldsNot asked forAsked for, so validate anyway
Groq's prompt ruleThe word json must appearNone needed in these examples

Where you use structured output

  • Sorting incoming text. Tickets, emails or reviews become a category plus a flag your code routes on.
  • Extracting fields. Names, dates and amounts pulled out of a paragraph into a typed object.
  • Feeding another system. A reply that goes straight into a database row or an API call.
Watch out. A schema asks the model for a shape. It does not guarantee it, and a provider can still return something that fails validation. Always parse the reply with the same class, catch ValidationError, and decide what to do on a miss.
Try it yourself
  • Add a field summary: str to Ticket and read the extra field in the printed object.
  • Change category: str to category: Literal["billing", "shipping", "other"] (import Literal from typing) and send a message about a lost parcel.
  • Add the word JSON to the json_object prompt and run it again to see the error go away.

This is what real progress feels like.