LLMOpsThe LLMOps lifecycle · MLflow 3 · FastAPI · Docker · AWS/Azure
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
41 small wins to finish your pathNext lesson →

Pick the build: RAG, fine-tuning, or both

The build decision is whether to give a model your knowledge at query time with RAG, bake it into the weights with fine-tuning, or do both with RAFT.

Last updated: 15 Oct, 2026 · LLMOps

The shape of an LLM pipeline showed where the model and the data sit in the flow. This lesson is the decision at the build stage of the lifecycle: how your own knowledge reaches the model. The video lays out three routes.

RAG, fine-tuning, or RAFT · from the Detailed LLMOps Project Lifecycle video · 6:40 to 7:26

The three routes

The lifecycle video describes each route by what it does to the model and the data.

  • RAG. You convert your documents into vectors, store them in a vector database, and retrieve the right passage at query time so the model answers from it. The model's weights never change; you change what you put in front of it.
  • Fine-tuning. You adjust the model itself on your data, which the video describes as retraining or fine-tuning the LLM so it behaves the way your company wants. The work is a training run: hyperparameter tuning, a training pipeline, and experiment tracking.
  • RAFT, which is both. The video names the combination of fine-tuning with RAG. You fine-tune a model and also retrieve from your documents, so the model learns your style while still reading current, specific facts.
A decision diagram. From the question what does the app need, three branches: needing current facts from your own documents leads to RAG, which is the lower cost to start because adding a document keeps answers current; needing a fixed voice, format or behaviour leads to fine-tuning, which costs more because it is a training run on a GPU and freezes the data into the weights; and needing both at once leads to RAFT, the highest cost, a fine-tuned model that also retrieves at query time.

The tradeoffs that decide it

Three tradeoffs separate the routes. RAG is cheaper to start and keeps answers current, because adding a document updates knowledge with no training. Fine-tuning changes how the model answers, which RAG cannot do, but costs a training run and goes stale as your data moves on. RAFT buys both and pays for both.

RAGFine-tuningRAFT (both)
Changes the modelNoYesYes
Cost to startLowerHigher (a training run)Highest
Keeps answers currentYes, add a documentNo, data is frozen inYes, through retrieval
Controls the answering styleWeakly, through the promptStronglyStrongly
Fits whenKnowledge changes oftenBehaviour must changeYou need both

Where you use each route

  • RAG, for a support assistant that must answer from policies and docs that change, where you cannot retrain every week.
  • Fine-tuning, when the model must answer in a fixed voice or format a prompt cannot hold reliably.
  • RAFT, when a project needs the fixed voice of fine-tuning and the current facts of retrieval at once.
Watch out. Fine-tuning needs a GPU and a training dataset, and this course shows it rather than running it. Most teams reach for RAG first because it is cheaper and keeps knowledge current, and only fine-tune once a clear need for changed behaviour appears. Data for fine-tuning covers what that training set has to look like.
Try it yourself
  • For a shop assistant that answers from a changing return policy, decide which route fits and say why in one line.
  • Name one requirement that would push you from RAG to fine-tuning.
  • Read The RAG data pipeline to see the data work RAG needs, then skim Data for fine-tuning for the training-set side.
PreviousThe shape of an LLM pipeline

Slow is fine. Stopping is the only problem.