model_validate_json: checking a model's answer

Lesson 18 caught answers that were not JSON. An answer can also be perfect JSON with the wrong values in it. A Pydantic model catches both in one step.

Examplethe Triage model from lesson 24

from typing import Literal
from pydantic import BaseModel, Field

class Triage(BaseModel):
    category: Literal["billing", "shipping", "other"]
    priority: int = Field(ge=1, le=5)

Examplefrom lesson 17

import json

def ask_model(ticket_text):
    text = ticket_text.lower()
    if "charged" in text or "refund" in text:
        return '{"category": "billing", "priority": 4}'
    if "parcel" in text or "arrived" in text:
        return '{"category": "shipping", "priority": 3}'
    return "I am not sure how to sort this one."

with open("tickets.json") as f:
    tickets = json.load(f)

Example

answer = ask_model("I was charged twice for one order")
result = Triage.model_validate_json(answer)
print(result)
print(result.category)

model_validate_json takes the answer text, reads it as JSON, and builds a Triage from it, checking every field on the way. It replaces json.loads plus your own checks.

Two ways an answer can be wrong

Example

from pydantic import ValidationError

for answer in ["I am not sure how to sort this one.", '{"category": "sales", "priority": 9}']:
    try:
        Triage.model_validate_json(answer)
    except ValidationError as error:
        print(error)
        print("---")

The first is not JSON at all, and the error says Invalid JSON. The second is valid JSON that breaks both rules. Either way it is one exception, ValidationError, so one except handles both.

Every ticket, checked

Example

checked = []
for ticket in tickets:
    try:
        result = Triage.model_validate_json(ask_model(ticket["text"]))
    except ValidationError:
        print(ticket["id"], "needs a person")
        continue
    checked.append(result)
    print(ticket["id"], result.category, result.priority)

print(len(checked), "checked")

Telling the model the shape

Many model APIs accept a JSON Schema, a description of the shape an answer must have, and use it to steer the model. Pydantic writes one from your class:

Example

import json
print(json.dumps(Triage.model_json_schema(), indent=2))

The allowed categories and the 1 to 5 limit are all in there. When a framework says it supports structured output with a Pydantic class, this schema is what it sends.

Try it yourself

Make ask_model return priority 7 for shipping tickets and run the loop again.
Print error.errors() instead of error inside the except.
Add "account" to the Literal and a password branch to ask_model.

PreviousPydantic models: data that checks itself

Every expert started right here.