LLM FundamentalsQwen2.5-0.5B-Instruct · transformers 5.17 · Python 3.10+
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
18 small wins to finish your path

The ticket sorter, measured

Lesson 0 promised a ticket-sorting prompt, a score for how often it is right, and its cost and speed. One script reports all three, on the real model.

One improvement goes in first. Lesson 11 left the password ticket wrong, and its exercise suggested an example about logins. The prompt file adds exactly that, a fourth example, and the score decides whether it helped.

The prompt, in its own file

Exampleprompt.py
from typing import Literal

from pydantic import BaseModel, Field, ValidationError


class Triage(BaseModel):
    category: Literal["billing", "shipping", "other"]
    priority: int = Field(ge=1, le=5)


def check(answer):
    try:
        return Triage.model_validate_json(answer)
    except ValidationError:
        return None


SYSTEM = """You sort customer support tickets for an online shop.

Categories:
- billing: payments, charges, refunds
- shipping: parcels, delivery, damaged boxes
- other: anything else

Reply with one line of JSON and nothing else, like this:
{"category": "shipping", "priority": 3}
priority is 1 (can wait) to 5 (urgent)."""


EXAMPLES = [
    {"role": "user", "content": "You took money from my card two times"},
    {"role": "assistant", "content": '{"category": "billing", "priority": 4}'},
    {"role": "user", "content": "Where is my delivery? It is a week late"},
    {"role": "assistant", "content": '{"category": "shipping", "priority": 3}'},
    {"role": "user", "content": "Can I change the email on my account?"},
    {"role": "assistant", "content": '{"category": "other", "priority": 2}'},
    {"role": "user", "content": "I forgot my login details"},
    {"role": "assistant", "content": '{"category": "other", "priority": 2}'},
]


TICKETS = [
    ("I was charged twice for one order", "billing"),
    ("My parcel has not arrived", "shipping"),
    ("Can I get a refund for the blue mug?", "billing"),
    ("How do I change my password?", "other"),
    ("The parcel arrived but the box was crushed", "shipping"),
]

The Triage model and check from lesson 12, the structured system prompt from lesson 10, the examples from lesson 11 plus the new login example, and the labelled tickets. Keeping the prompt apart from the code means a change to the prompt is a change to one file, easy to review and score.

The script

Examplesort.py, part 1
import time

from transformers import AutoModelForCausalLM, AutoTokenizer

from prompt import check, EXAMPLES, SYSTEM, TICKETS

name = "Qwen/Qwen2.5-0.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name)
Examplesort.py, part 2
def sort_ticket(text):
    messages = [{"role": "system", "content": SYSTEM}, *EXAMPLES, {"role": "user", "content": text}]
    ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True)
    output = model.generate(**ids, max_new_tokens=30, do_sample=False)
    new = output[0][ids["input_ids"].shape[1]:]
    return tokenizer.decode(new, skip_special_tokens=True), ids["input_ids"].shape[1], len(new)

return_dict=True gives the ids and the attention mask in one dictionary, ready for generate. The function returns the answer and both token counts, which the cost line needs.

Examplesort.py, part 3
right, tokens_in, tokens_out = 0, 0, 0
started = time.perf_counter()
for text, expected in TICKETS:
    answer, used_in, used_out = sort_ticket(text)
    tokens_in, tokens_out = tokens_in + used_in, tokens_out + used_out
    result = check(answer)
    ok = result is not None and result.category == expected
    right += ok
    print(f"{'ok ' if ok else 'BAD'} {expected:9} {answer}")
seconds = time.perf_counter() - started

print(f"accuracy {right}/{len(TICKETS)}")
print(f"tokens   {tokens_in} in, {tokens_out} out")
print(f"cost     ${(tokens_in * 0.50 + tokens_out * 1.50) / 1_000_000:.6f} at the example prices")
print(f"time     {seconds / len(TICKETS):.2f} seconds a ticket")

Each ticket is sorted, checked (lesson 12) and scored against its label (lesson 13). Token counts feed the cost at the example prices (lesson 15), and the clock gives time per ticket (lesson 16). right += ok adds 1 for True.

Run it

Example
python sort.py

Five of five: the login example fixed the password ticket, and nothing else broke. It also added tokens to every call, which the cost line already includes.

That is the whole loop an AI engineer runs on a prompt: change one thing, score it, look at the cost and the time, keep it or throw it away. The next step is more labelled tickets, because five cannot tell a real improvement from luck.

Where each piece came from

sort.py, by lesson
The modelloading, lesson 1chat templates, lesson 9greedy decoding, lesson 4The promptsystem prompt, lesson 10examples, lesson 11CheckingPydantic, lesson 12scoring, lesson 13Productiontokens and cost, lesson 15time, lesson 16sort.py

Choosing a model

The same script answers the question every project asks: is a small model good enough? Change name to a larger model, "Qwen/Qwen2.5-1.5B-Instruct" is three times the size, and compare the three numbers. A hosted model is the same comparison with the generate call replaced by an API call from APIs for AI. Pick the cheapest model whose score clears the bar you agreed on, not the biggest one available.

What this course left out

TopicWhat it is for
EmbeddingsTurning text into vectors to find similar documents, the base of retrieval.
Tool callingLetting the model ask your code to run a function, the base of agents.
Fine-tuningTraining a model's weights on your own examples when prompting is not enough.
QuantizationStoring weights in fewer bits so larger models fit in less memory.
Reasoning modelsModels that write out intermediate steps before answering, trading tokens and time for accuracy.
Safety and prompt injectionText in a ticket that tries to override your instructions; covered in the guardrails and red-teaming courses.

You understood something today that you didn't yesterday.