What you are going to build

You will finish this course with a prompt that sorts support tickets, a score saying how often it is right, and the cost and speed of running it, measured on a real model on your own computer.

Most explanations of large language models stop at diagrams. Here every idea is something you run: you will look at the probabilities a model gives each possible next word, change the temperature and watch the answers change, and see a prompt go from wrong to mostly right.

The model you will run

Qwen2.5-0.5B-Instruct, a small open model from Alibaba's Qwen team, released under the Apache 2.0 licence. It is about 1 GB to download and answers in a second or two on a laptop's CPU, with no GPU and no API key.

It is small, and it makes mistakes a large hosted model would not. That is useful here: a prompt that fixes a mistake you can see teaches more than one that polishes an answer that was already right. Everything you learn applies unchanged to the largest models.

What you need

• Python 3.10 or later, about 3 GB of free disk space, and 4 GB of free memory.
• Lists, dictionaries, functions and Pydantic, from Python for AI.

pip install "torch==2.14.0" "transformers==5.17.0" "tiktoken==0.14.0" pydantic

torch does the maths, transformers loads and runs models from Hugging Face, and tiktoken is OpenAI's tokenizer, used in lesson 15. The model itself downloads the first time lesson 1 runs.