Structured output your code can read
Structured output is a request option that makes a model reply with JSON that fits a schema you give it, so your code can read the reply as data instead of parsing prose.
Last updated: 12 Oct, 2026 · LiteLLM 1.105
Tool calling: when the model asks for data had the model send arguments as JSON. This lesson asks for the whole reply in a shape you choose, such as a support ticket with a category and an urgency flag. LiteLLM takes the schema in one form and passes it to each provider in the form that provider expects.
Syntax: pass a Pydantic class (or a JSON schema dictionary) as response_format, then validate the reply text with the same class.
response = completion(
model="groq/openai/gpt-oss-20b",
messages=messages,
response_format=Ticket, # the shape the reply must have
max_tokens=300,
)
ticket = Ticket.model_validate_json(response.choices[0].message.content)Describing the reply with a Pydantic class
The schema as a class
Each attribute becomes a field the reply must contain, and its type becomes the type of the value. LiteLLM turns the class into a JSON schema before sending it.
from pydantic import BaseModel
class Ticket(BaseModel):
category: str # billing, shipping or other
urgent: boolChecking that a model supports schemas
Not every model accepts a schema. litellm.supports_response_schema() answers from LiteLLM's model table, without a call.
import litellm
for model in ["groq/openai/gpt-oss-120b", "groq/openai/gpt-oss-20b", "gemini/gemini-2.5-flash-lite"]:
print(model, litellm.supports_response_schema(model))groq/openai/gpt-oss-120b True groq/openai/gpt-oss-20b True gemini/gemini-2.5-flash-lite True
Reading the reply as an object
The reply text is JSON. model_validate_json parses it and checks every field in one step, and it returns a Ticket, not a dictionary.
from pydantic import BaseModel
class Ticket(BaseModel):
category: str # billing, shipping or other
urgent: bool
from litellm import completion
messages = [{"role": "user", "content":
"Classify this message. Category is billing, shipping or other: "
"My card was charged twice, fix it today!"}]
for model in ["groq/openai/gpt-oss-20b", "gemini/gemini-2.5-flash-lite"]:
response = completion(model=model, messages=messages, response_format=Ticket, max_tokens=300)
text = response.choices[0].message.content
print(repr(text))
print(Ticket.model_validate_json(text))'{"category":"billing","urgent":true}'
category='billing' urgent=True
'{\n "category": "billing",\n "urgent": true\n}'
category='billing' urgent=True- The text differs, the data does not. The text can differ in spacing: in this run Gemini spread the JSON over several lines and gpt-oss returned one compact line. Both parse to
category='billing' urgent=True. - Validate, do not compare strings. Spacing and key order are the provider's choice, so code that matches the text exactly breaks on a provider change.
What happens without a schema
Before schemas, the usual request was {"type": "json_object"}, which asks for valid JSON of any shape. Groq adds a rule to it: the word json must appear somewhere in the messages.
import litellm
from litellm import completion
litellm.suppress_debug_info = True # hides LiteLLM's help banner on errors
try:
completion(
model="groq/openai/gpt-oss-20b",
messages=[{"role": "user", "content": "Classify: My card was charged twice."}],
response_format={"type": "json_object"},
max_tokens=300,
)
except litellm.BadRequestError as error:
print(type(error).__name__)
print(error)BadRequestError
litellm.BadRequestError: GroqException - {"error":{"message":"'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'.","type":"invalid_request_error"}}LiteLLM raised BadRequestError and carried Groq's message in the error text. The schema form in the earlier examples needed no such word. Even with the word added, json_object promises valid JSON, not your fields.
Catching a reply that does not fit
A reply that is valid JSON but missing a field passes json.loads and fails later, at the line that needs the field. Validating with the class fails at once and names the field.
from pydantic import BaseModel
class Ticket(BaseModel):
category: str # billing, shipping or other
urgent: bool
from pydantic import ValidationError
try:
Ticket.model_validate_json('{"category": "billing"}')
except ValidationError as error:
print([(item["loc"], item["type"]) for item in error.errors()])[(('urgent',), 'missing')]The error names the missing field, urgent, and its type, missing. A program can catch it and ask the model again.
json_object vs a schema
| json_object | A schema (Pydantic class) | |
|---|---|---|
| You send | {"type": "json_object"} | The class or a JSON schema dictionary |
| Reply is valid JSON | Asked for | Asked for |
| Reply has your fields | Not asked for | Asked for, so validate anyway |
| Groq's prompt rule | The word json must appear | None needed in these examples |
Where you use structured output
- Sorting incoming text. Tickets, emails or reviews become a category plus a flag your code routes on.
- Extracting fields. Names, dates and amounts pulled out of a paragraph into a typed object.
- Feeding another system. A reply that goes straight into a database row or an API call.
ValidationError, and decide what to do on a miss.Related
- Previous: Tool calling: when the model asks for data
- Next: Multimodal: pictures and sound in the same call
- See also: Exceptions: one set for every provider for the errors a rejected request raises
- Reference: Structured outputs (JSON mode) in the LiteLLM docs
- Add a field
summary: strtoTicketand read the extra field in the printed object. - Change
category: strtocategory: Literal["billing", "shipping", "other"](importLiteralfromtyping) and send a message about a lost parcel. - Add the word
JSONto thejson_objectprompt and run it again to see the error go away.
This is what real progress feels like.