Fine-tuning LLMs
Fine-tuning is a training step that continues from a pre-trained model and adjusts its weights on a smaller, task-specific set of examples, so the model's behaviour, style or format changes.
Last updated: 08 Oct, 2026 · NumPy
A pre-trained model has learned language from a very large text collection. Fine-tuning shows it examples of the behaviour you want, as input and output pairs, and trains on them. For language models this is called supervised fine-tuning (SFT): the loss is the next-token cross-entropy on the target text.
Base models and the ways of fine-tuning them
A base or pre-trained model, such as GPT-3.5, GPT-4 or Llama 2, has been trained on web pages, books and other sources to predict the next token. The clip separates three ways of fine-tuning it:
| Kind | What is trained | Example |
|---|---|---|
| Full-parameter fine-tuning | Every weight of the model | A general chat assistant built from a base model |
| Domain-specific fine-tuning | The model, on text from one field | A finance chatbot, a sales assistant |
| Task-specific fine-tuning | The model, on one kind of job | Text to SQL, question answering on documents |
Domain and task describe the data. Full-parameter describes how much of the model is trained. The two combine: a task can be taught by training every weight or only a few, and the second is the subject of LoRA and QLoRA. Training only a small part of a model is called parameter-efficient fine-tuning, PEFT.
Why full fine-tuning is expensive
Training every weight means holding more than the weights. With the Adam optimizer in 32-bit, each trained parameter needs its weight, its gradient and Adam's two running averages: four numbers of 4 bytes. The next block counts that for Llama 2 7B and compares it with LoRA and QLoRA, which the next lesson builds.
n = 6_738_415_616 # Llama-2 7B parameters
adapter = 33_554_432 # r = 64 on q_proj and v_proj
# bytes per trained parameter: weight + gradient + two Adam moments, all FP32
full = n * (4 + 4 + 4 + 4)
print(f"full fine-tuning, FP32 Adam : {full / 1e9:7.1f} GB before activations")
# LoRA on an FP16 base: base weights frozen, only the adapters get gradients and Adam state
lora = n * 2 + adapter * 16
print(f"LoRA on an FP16 base : {lora / 1e9:7.1f} GB")
# QLoRA: base in 4 bits (+ 0.127 extra bits per weight with double quantization)
qlora = 6_476_005_376 * (4 + 0.127) / 8 + (n - 6_476_005_376) * 2 + adapter * 16
print(f"QLoRA, 4-bit base : {qlora / 1e9:7.1f} GB")
print("free Colab T4 GPU : 15.0 GB")full fine-tuning, FP32 Adam : 107.8 GB before activations LoRA on an FP16 base : 14.0 GB QLoRA, 4-bit base : 4.4 GB free Colab T4 GPU : 15.0 GB
Reading the memory counts
- Full fine-tuning needs 107.8 GB before activations are counted: 16 bytes for each of 6,738,415,616 parameters. A free Colab T4 GPU has 15 GB.
- LoRA on an FP16 base needs 14.0 GB. The base is frozen, so only the 33,554,432 adapter values carry gradients and optimizer state, and the weights dominate.
- QLoRA needs 4.4 GB. The linear layers of the base are 4-bit with double-quantized scales, and the rest of the model stays 16-bit.
Fine-tuning, prompting and retrieval
Fine-tuning changes the weights. Prompting and RAG change what the model is shown. They solve different problems, so the order that works is the cheapest first. Retrieval is built in Retrieval-augmented generation (RAG).
| Prompting | RAG | Fine-tuning | |
|---|---|---|---|
| What changes | The instructions in the prompt | The documents in the prompt | The model's weights |
| Good for | Format, tone, a few examples | Facts that change, private documents, citations | A fixed style or format, a narrow task, a smaller model that does one job well |
| Needs | Nothing to train | An index of documents | Hundreds to thousands of good examples and a GPU run |
| Updating | Edit the prompt | Re-index the documents | Train again |
| Weak at | A behaviour the model will not follow however it is told | Changing how the model writes | Adding facts that stay correct |
A model fine-tuned on facts stores them loosely and can state them wrongly; keeping facts in documents and retrieving them is the dependable route. Fine-tuning earns its cost when the behaviour is wrong even with a good prompt and good context: a strict output format, a house style, a domain's vocabulary, or a small model that must do one task as well as a larger one.
Where you use fine-tuning
- Output format. A model that must always answer in one JSON shape or one template.
- Tone and style. A support assistant that writes in the company's voice.
- A narrow task on a small model. Text to SQL or classification, at lower cost per call than a large model.
Related
- Previous: 1-bit LLMs (BitNet b1.58)
- Next: LoRA and QLoRA
- See also: Retrieval-augmented generation (RAG)
- See also: How LLMs are built
- In the memory block, change the adapter count to 159,907,840 (r = 64 on all linear layers) and read how the LoRA rows move.
- Replace the 16 bytes of full fine-tuning with 8 (a 16-bit optimizer state) and read the total again.
- Write down one task of yours and decide for each column of the comparison table whether it solves it.
Little by little, you're building something great.