Generative AI and Agentic AILangChain 1.4 · LangGraph 1.2 · Python 3.12 or 3.13
Dashboard
0%

Fine-tuning LLMs

Fine-tuning is a training step that continues from a pre-trained model and adjusts its weights on a smaller, task-specific set of examples, so the model's behaviour, style or format changes.

Last updated: 08 Oct, 2026 · NumPy

A pre-trained model has learned language from a very large text collection. Fine-tuning shows it examples of the behaviour you want, as input and output pairs, and trains on them. For language models this is called supervised fine-tuning (SFT): the loss is the next-token cross-entropy on the target text.

Base models and the ways of fine-tuning them

Base models and the ways of fine-tuning · from the Fine-Tuning LLM Models Crash Course video · 36:08 to 39:54

A base or pre-trained model, such as GPT-3.5, GPT-4 or Llama 2, has been trained on web pages, books and other sources to predict the next token. The clip separates three ways of fine-tuning it:

KindWhat is trainedExample
Full-parameter fine-tuningEvery weight of the modelA general chat assistant built from a base model
Domain-specific fine-tuningThe model, on text from one fieldA finance chatbot, a sales assistant
Task-specific fine-tuningThe model, on one kind of jobText to SQL, question answering on documents

Domain and task describe the data. Full-parameter describes how much of the model is trained. The two combine: a task can be taught by training every weight or only a few, and the second is the subject of LoRA and QLoRA. Training only a small part of a model is called parameter-efficient fine-tuning, PEFT.

Three steps from cheapest to most expensive: prompting, then retrieval-augmented generation, then fine-tuning, each with the problem it solves.

Why full fine-tuning is expensive

Training every weight means holding more than the weights. With the Adam optimizer in 32-bit, each trained parameter needs its weight, its gradient and Adam's two running averages: four numbers of 4 bytes. The next block counts that for Llama 2 7B and compares it with LoRA and QLoRA, which the next lesson builds.

ExampleTraining memory for a 7B model, weights, gradients and optimizer state
n = 6_738_415_616                                     # Llama-2 7B parameters
adapter = 33_554_432                                  # r = 64 on q_proj and v_proj

# bytes per trained parameter: weight + gradient + two Adam moments, all FP32
full = n * (4 + 4 + 4 + 4)
print(f"full fine-tuning, FP32 Adam : {full / 1e9:7.1f} GB before activations")

# LoRA on an FP16 base: base weights frozen, only the adapters get gradients and Adam state
lora = n * 2 + adapter * 16
print(f"LoRA on an FP16 base       : {lora / 1e9:7.1f} GB")

# QLoRA: base in 4 bits (+ 0.127 extra bits per weight with double quantization)
qlora = 6_476_005_376 * (4 + 0.127) / 8 + (n - 6_476_005_376) * 2 + adapter * 16
print(f"QLoRA, 4-bit base          : {qlora / 1e9:7.1f} GB")
print("free Colab T4 GPU          :    15.0 GB")

Reading the memory counts

  • Full fine-tuning needs 107.8 GB before activations are counted: 16 bytes for each of 6,738,415,616 parameters. A free Colab T4 GPU has 15 GB.
  • LoRA on an FP16 base needs 14.0 GB. The base is frozen, so only the 33,554,432 adapter values carry gradients and optimizer state, and the weights dominate.
  • QLoRA needs 4.4 GB. The linear layers of the base are 4-bit with double-quantized scales, and the rest of the model stays 16-bit.

Fine-tuning, prompting and retrieval

Fine-tuning changes the weights. Prompting and RAG change what the model is shown. They solve different problems, so the order that works is the cheapest first. Retrieval is built in Retrieval-augmented generation (RAG).

PromptingRAGFine-tuning
What changesThe instructions in the promptThe documents in the promptThe model's weights
Good forFormat, tone, a few examplesFacts that change, private documents, citationsA fixed style or format, a narrow task, a smaller model that does one job well
NeedsNothing to trainAn index of documentsHundreds to thousands of good examples and a GPU run
UpdatingEdit the promptRe-index the documentsTrain again
Weak atA behaviour the model will not follow however it is toldChanging how the model writesAdding facts that stay correct

A model fine-tuned on facts stores them loosely and can state them wrongly; keeping facts in documents and retrieving them is the dependable route. Fine-tuning earns its cost when the behaviour is wrong even with a good prompt and good context: a strict output format, a house style, a domain's vocabulary, or a small model that must do one task as well as a larger one.

Where you use fine-tuning

  • Output format. A model that must always answer in one JSON shape or one template.
  • Tone and style. A support assistant that writes in the company's voice.
  • A narrow task on a small model. Text to SQL or classification, at lower cost per call than a large model.
Watch out. The memory counts are lower bounds: activations, the attention cache and fragmentation add more, and a run that fits on paper can still run out of memory with long sequences. Cut the batch size or sequence length first.
Try it yourself
  • In the memory block, change the adapter count to 159,907,840 (r = 64 on all linear layers) and read how the LoRA rows move.
  • Replace the 16 bytes of full fine-tuning with 8 (a 16-bit optimizer state) and read the total again.
  • Write down one task of yours and decide for each column of the comparison table whether it solves it.

Little by little, you're building something great.