Guardrails AIguardrails-ai 0.11.0 · Python 3.10+
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
27 small wins to finish your pathNext lesson

When the JSON is not JSON

Lesson 19 asked the model for JSON properly. Models still get it wrong, and a structured Guard fails differently from a string one.

Example
from pretend_guardrails import PretendModel

model = PretendModel(replies=["Sure! Here you go: {'order_id': 8821, 'issue': 'late'}"])
tickets = Guard.for_pydantic(Ticket)

outcome = tickets(model, messages=[{"role": "user", "content": "File a ticket."}], num_reasks=0)

print(outcome.validation_passed)
print(outcome.error)
print(outcome.reask.fail_results[0].error_message)

Single quotes are not JSON. The extraction step from lesson 17 found something that looked like an object, handed it to the JSON parser, and the parser refused at the first character it did not recognise.

Two places record it. outcome.error carries the parser's own complaint, which is the one that tells you where. outcome.reask carries the Guardrails version, which is the one that would have gone back to the model.

Letting it try again

Example
model = PretendModel(replies=[
    "Sure! Here you go: {'order_id': 8821, 'issue': 'late'}",
    '{"order_id": "8821", "issue": "late", "priority": 2}',
])
tickets = Guard.for_pydantic(Ticket)

outcome = tickets(model, messages=[{"role": "user", "content": "File a ticket."}], num_reasks=1)

print(outcome.validation_passed)
print(outcome.validated_output)
print("model calls:", len(model.prompts))

No on_fail anywhere in that snippet. A structural failure reasks on its own when num_reasks allows it, which is different from lesson 16, where reasking was a choice a validator made.

Example
print(model.prompts[1][-1]["content"][:420])

The second prompt carries the schema whether or not you put the placeholder in the first one. Guardrails knows the shape it wanted; it simply does not volunteer it until something has gone wrong.

Which is a reasonable default and an expensive one. A first prompt with the placeholder costs tokens once. A first prompt without it costs a wasted call every time the model guesses badly.

Try it yourself
  • Script a second reply that is also broken and watch the reask budget run out.
  • Return valid JSON with priority as the word high and see which of the two failure kinds you get.
  • Set num_reasks=2 and script three replies. Count the iterations in the history.

Every expert started right here.