System prompts: saying exactly what you want

The course's task starts here: sort each ticket into billing, shipping or other, with a priority, as JSON. A vague instruction goes first, and fails.

The examples use reply from lesson 9.

Examplefive tickets, with the right category for each

tickets = [
    ("I was charged twice for one order", "billing"),
    ("My parcel has not arrived", "shipping"),
    ("Can I get a refund for the blue mug?", "billing"),
    ("How do I change my password?", "other"),
    ("The parcel arrived but the box was crushed", "shipping"),
]

Example

vague = "Sort this support ticket into billing, shipping or other and give it a priority."

Example

for text, expected in tickets:
    answer = reply([{"role": "system", "content": vague}, {"role": "user", "content": text}])
    print(f"{expected:9} {answer!r}")

The instruction never said what shape to answer in, so the model chose, and different tickets got different shapes. None of these can be read by a program.

A structured prompt

Example

structured = """You sort customer support tickets for an online shop.

Categories:
- billing: payments, charges, refunds
- shipping: parcels, delivery, damaged boxes
- other: anything else

Reply with one line of JSON and nothing else, like this:
{"category": "shipping", "priority": 3}
priority is 1 (can wait) to 5 (urgent)."""

It says who the model is, defines each category with examples of what belongs, and shows the exact output with one sample line. Triple quotes let a string run over several lines.

Example

for text, expected in tickets:
    answer = reply([{"role": "system", "content": structured}, {"role": "user", "content": text}])
    print(f"{expected:9} {answer}")

Every answer is now JSON in the right shape. Look at the categories, though: they are all the same. The format instruction worked and the sorting did not, which is a common result with small models and happens with large ones on harder tasks.

Valid JSON is not a correct answer. Lesson 12 checks the shape automatically, and lesson 13 measures correctness, because reading five lines by eye does not scale to five thousand.

Try it yourself

Remove the sample JSON line from structured and run it again.
Put the categories in the user message instead of the system message.
Add "If unsure, use other." to the prompt and see which answers change.

PreviousChat templates: how messages become one text

Every expert started right here.