Examples: labelled data for DSPy

dspy.Example holds one data point as named fields. with_inputs says which fields go into the program; the rest are labels to compare against.

Example

example = dspy.Example(ticket="I was charged twice", category="billing").with_inputs("ticket")
print(example.ticket, example.category)
print(example.inputs())
print(example.labels())

An Example works like a dictionary with attribute access. with_inputs("ticket") marks ticket as the input; inputs() and labels() split the fields. Evaluation and optimizers call your program with **example.inputs() and compare the answer with the labels, using a metric, lesson 13.

The tickets for the rest of the course

Exampletickets.py

import dspy

ROWS = [
    ("I was charged twice for one order", "billing"),
    ("My parcel never arrived", "shipping"),
    ("I forgot my password", "account"),
    ("I want my money back for the broken lamp", "billing"),
    ("The courier left the box at the wrong house", "shipping"),
    ("How do I change the email on my account?", "account"),
    ("My card was declined but the money left my bank", "billing"),
    ("Tracking has said in transit for ten days", "shipping"),
    ("Someone else is signed in to my profile", "account"),
    ("Can I get an invoice for my company?", "billing"),
    ("The box came without the charger", "shipping"),
    ("Please delete my profile and my data", "account"),
    ("Send my money back, the lamp was broken", "billing"),
    ("Tracking still shows in transit after two weeks", "shipping"),
    ("I cannot sign in to my profile", "account"),
    ("You took the money twice from my card", "billing"),
    ("The box was left at the wrong door", "shipping"),
    ("Change my profile email please", "account"),
    ("Where is my money back for the returned lamp?", "billing"),
    ("My box is stuck at customs", "shipping"),
]

examples = [dspy.Example(ticket=t, category=c).with_inputs("ticket") for t, c in ROWS]
trainset, devset = examples[:12], examples[12:]

Example

print(len(trainset), len(devset))
print(devset[0])
print(sum(e.category == "billing" for e in devset), "billing tickets in devset")

Twenty tickets: 12 in trainset for optimizers to learn from, 8 in devset to measure on. Many devset tickets are worded like a training ticket, money back, tracking, profile, but use none of the stand-in's keywords.

Keep the sets apart

A score measured on tickets the optimizer learned from says how well it memorised them, not how it handles new ones. Real projects keep a third set, the test set, untouched until the end.

Try it yourself

Add a ticket of your own to ROWS and print len(devset).
Create an example with with_inputs("ticket", "category") and print labels().
Print example.toDict().

PreviousBestOfN: try again until a check passes

You understood something today that you didn't yesterday.