Writing the stand-in model

Everything so far checked a string you already had. The other half of Guardrails wraps the model call itself, and that needs a model. This lesson writes one that costs nothing, and writing it is how you learn what Guardrails asks a model for.

The whole interface

Guardrails reaches a model in two ways. Name a model with model="gpt-4o" and it goes through LiteLLM, which needs a key. Hand it a callable instead and it uses that. The Use supported LLMs page describes the callable in one sentence: it takes keyword arguments and returns the model output as a string.

Rather than trust that, print it.

Example

from guardrails import Guard
from guardrails_ai.valid_length import ValidLength


def spy(**kwargs):
    print("keyword arguments:", sorted(kwargs))
    print("messages:", kwargs["messages"])
    return "Order 8821 ships today."


desk = Guard().use(ValidLength(min=1, max=40, on_fail="noop"))
desk(spy, messages=[{"role": "user", "content": "Where is order 8821?"}])

Two keyword arguments arrive: the messages list you passed in, untouched, and a temperature Guardrails fills in. Whatever the function returns is what the validators see. That is the entire contract, and it is why this course needs no key.

A model that decides

A stand-in that always says the same thing teaches you the plumbing and nothing else. This one reads the question and picks an answer, so the early lessons show a choice being made.

python

import re

ANSWERS = [
    ("refund", "We can refund order {id} in full, today."),
    ("late", "Order {id} is running late and should arrive on Friday."),
    ("where", "Order {id} left the warehouse this morning."),
]
FALLBACK = "I do not have that on file. Please send your order number."

The table the model answers from, and the sentence it falls back to when nothing matches.

Example

def order_id(text):
    found = re.findall(r"\b\d{4}\b", text)
    return found[0] if found else "0000"


print(order_id("Where is order 8821?"), "|", order_id("hello"))

Three keywords and a regular expression that pulls a four digit order number out of the question. Nothing clever, and that is the point: the answer depends on what was asked.

The class wraps that in the callable shape the spy just showed.

python

"""A stand-in model for Guardrails that needs no API key.

Guardrails reaches a model through a callable. It hands that callable a
`messages` keyword argument and expects a string back, so a stand-in is an
ordinary Python object with a `__call__`. This one answers support questions
from a small table, and can also be handed a scripted list of replies when a
lesson needs to control exactly what the model says.
"""
import re

ANSWERS = [
    ("refund", "We can refund order {id} in full, today."),
    ("late", "Order {id} is running late and should arrive on Friday."),
    ("where", "Order {id} left the warehouse this morning."),
]
FALLBACK = "I do not have that on file. Please send your order number."
REASK_MARK = "Generate a new response that corrects your old response"


def order_id(text):
    """The first four digit number in the question, or 0000."""
    found = re.findall(r"\b\d{4}\b", text)
    return found[0] if found else "0000"


class PretendModel:
    """Decides a reply from the question, unless a reply was scripted."""

    def __init__(self, replies=None):
        self.replies = list(replies or [])
        self.prompts = []

    def __call__(self, **kwargs):
        messages = kwargs.get("messages") or []
        self.prompts.append(messages)
        asked = messages[-1]["content"] if messages else ""
        if self.replies:
            return self.replies.pop(0)
        if REASK_MARK in asked:
            return "Sorry, let me try that again."
        for word, reply in ANSWERS:
            if word in asked.lower():
                return reply.format(id=order_id(asked))
        return FALLBACK

Save it as pretend_guardrails.py next to your lessons. Two things in it matter later. It keeps every prompt it was handed in self.prompts, which is how lessons 16 and 19 print what Guardrails puts in front of a model. And it accepts a list of scripted replies, so a lesson that needs an exact answer can say so.

Example

from pretend_guardrails import PretendModel

model = PretendModel()

print(model(messages=[{"role": "user", "content": "Where is order 8821?"}]))
print(model(messages=[{"role": "user", "content": "Can I get a refund on 8821?"}]))
print(model(messages=[{"role": "user", "content": "hello"}]))

Example

from pretend_guardrails import PretendModel

scripted = PretendModel(replies=["We can refund order 8821 in full, today."])

print(scripted(messages=[{"role": "user", "content": "anything at all"}]))
print(scripted(messages=[{"role": "user", "content": "anything at all"}]))

The first call takes the scripted reply. The second finds the list empty and falls back to deciding, which is why the same question gives two different answers. Lessons that need certainty script every reply they depend on.

Try it yourself

Add a fourth row to ANSWERS for the word cancel and check that the desk answers it.
Make order_id return None when there is no number, and decide what the reply should say then.
Print model.prompts after three calls. That list is the whole record of what the Guard sent.

PreviousA function as on_fail

Slow is fine. Stopping is the only problem.