Curious builder0 XP earned · 300 to level 2

0 daysFinish a lesson to begin

Badge collection0 of 6 unlocked

24 small wins to finish your path

Optimized ticket desk, end to end

The final project joins a two-step desk module, a metric, an optimizer run that saves its result, an app that loads it, and tests that need no model.

The program

Exampledesk.py

from typing import Literal

import dspy


class Triage(dspy.Signature):
    """Sort a support ticket for an online shop."""

    ticket: str = dspy.InputField()
    category: Literal["billing", "shipping", "account"] = dspy.OutputField()


class Desk(dspy.Module):
    def __init__(self):
        super().__init__()
        self.triage = dspy.Predict(Triage)
        self.reply = dspy.Predict("ticket, category -> reply")

    def forward(self, ticket):
        category = self.triage(ticket=ticket).category
        reply = self.reply(ticket=ticket, category=category).reply
        return dspy.Prediction(category=category, reply=reply)


def exact(example, prediction, trace=None):
    return example.category == prediction.category

Triage, Desk and exact are from lessons 5, 8 and 13. The metric only checks the category, the step that has labels.

Every predictor in a module gets demos from the same examples, so the examples now carry a reply too, for the reply step:

Exampletickets.py, the last lines

examples = [
    dspy.Example(ticket=t, category=c, reply=f"Our {c} team will help.").with_inputs("ticket")
    for t, c in ROWS
]

Optimize once

Exampleoptimize.py

import dspy

from desk import Desk, exact
from shop_lm import ShopLM
from tickets import devset, trainset

dspy.configure(lm=ShopLM(model="shop/keywords"))
evaluate = dspy.Evaluate(devset=devset, metric=exact, num_threads=1, display_progress=False)

program = Desk()
print("before:", evaluate(program).score)

compiled = dspy.LabeledFewShot(k=12).compile(program, trainset=trainset)
print("after:", evaluate(compiled).score)
for name, predictor in compiled.named_predictors():
    print(f"  {name}: {len(predictor.demos)} demos")

compiled.save("desk.json")
print("saved desk.json")

Example

python optimize.py

25.0 before and 87.5 after, the same numbers as for Triage alone in lesson 16, because the category comes from the triage step. Both predictors received the same twelve examples, and each prompt uses only the fields its own signature has. desk.json holds both demo lists.

Use the saved program

Exampleapp.py

import dspy

from desk import Desk
from shop_lm import ShopLM

dspy.configure(lm=ShopLM(model="shop/keywords"))

desk = Desk()
desk.load("desk.json")

for ticket in ["Where is my money back? The lamp came broken", "Tracking says in transit for a week"]:
    result = desk(ticket=ticket)
    print(ticket)
    print("  ", result.category, "|", result.reply)

Example

python optimize.py > /dev/null 2>&1 && python app.py

In a new folder the optimizer has to run once first, so desk.json exists; its output is hidden here. Neither ticket has a keyword, and both were sorted correctly because the loaded demos include similar tickets. The replies come from the stand-in's template, since it has no rule that copies a demo's reply. The app never compiles anything; it loads the file the optimizer wrote.

Optimize once, load the result

Hover or tap a piece to see what it is and which lesson built it.

Run a script

Running the desk's tests

Example

pytest -q

The tests from lesson 22 run unchanged against this desk.py.

Optimizing for a hosted model

Change the dspy.configure line in optimize.py and app.py to dspy.LM("openai/gpt-4o-mini") and set the key. Measure before optimizing: a real model will already sort most of these tickets, so the interesting question becomes whether demos or an instruction optimizer from lesson 19 still add enough to pay for their tokens.

Next experiments with the desk

Try it yourself

Swap LabeledFewShot for BootstrapFewShotWithRandomSearch with a valset, and compare on devset.
Add a metric for replies and optimize for both.
Serve app.py's desk from a FastAPI endpoint, as in APIs for AI.

DSPy features for later

Topic	What it is for
Retrieval	Modules that search documents, and dspy.Embedder for embeddings.
dspy.History	Multi-turn conversations as a typed input field.
Streaming and async	dspy.streamify and acall for serving programs.
MCP tools	Converting an MCP server's tools into dspy.Tool for ReAct.
BootstrapFinetune	Turning passing runs into fine-tuning data for a smaller model.
MLflow	Tracing every call and tracking optimizer runs.
Custom adapters	Subclassing Adapter for your own prompt format.

PreviousTesting DSPy programs with DummyLM

Slow is fine. Stopping is the only problem.