PixioLab

RAG vs fine-tuning: which one your feature needs

By PixioLab ·

Use retrieval (RAG) when the model needs facts it doesn't have, especially facts that change. Use fine-tuning when it needs to behave differently: a consistent format, a tone, or one narrow task done cheaply at volume. Most product features need retrieval first, and many never need fine-tuning at all.

What each one changes

RAG changes what the model sees. At request time, your code searches your documents, puts the best passages in the prompt, and asks the model to answer from them with citations. The model itself is untouched.

Fine-tuning changes the model itself. You train it further on your own examples, and its weights shift toward them. What it learns is baked in: it can't cite where an answer came from, and it doesn't update when your documents do.

That difference decides most cases. Knowledge that changes, must be cited, or depends on who's asking belongs in retrieval. Behavior that prompting can't hold steady is what fine-tuning is for.

When RAG is the right choice

  • The answers live in private or recent documents the model never saw.
  • The documents change, and answers must change with them: re-index a file and the next answer uses it.
  • Answers must cite their sources, so a reader can check them.
  • Different users may see different documents: filter by permission at retrieval time.
  • You need to remove a fact quickly: delete it from the index and it's gone.

When fine-tuning is the right choice

  • A strict format or style that prompts and examples can't hold reliably.
  • One narrow task, such as classification or extraction, at a volume where a small fine-tuned model is much cheaper than a large general one.
  • Teaching a smaller, faster model to copy a larger model's behavior on your task.

It needs a labeled set of good examples, usually hundreds or more, and an evaluation that proves the tuned model beats the prompted baseline. Without that comparison, you can't tell whether it helped.

The third option: long context

If your documents fit comfortably in the model's context window, the simplest design is to put them in the prompt. With prompt caching, the repeated part costs less and returns faster. It stops working when the documents outgrow the window, when every request would pay for far more text than it needs, or when users may only see some of the documents.

The order to try them in

  1. Write a clear prompt with a few good examples, and measure it on a small labeled set.
  2. If the model lacks knowledge, add it: long context when the documents are small and stable, RAG when they're large, changing or permissioned.
  3. Measure retrieval on its own before judging answers: if the right passage isn't retrieved, no prompt will save the answer.
  4. Fine-tune only for behavior the earlier steps can't hold, and keep the prompted version as the baseline to beat.

Where the course goes deeper

Day 15 of the roadmap weighs RAG against long context and fine-tuning before Week 3 builds retrieval step by step. RAG, Explained covers the pipeline in depth, and the Slotfill project in the portfolio catalog is about fine-tuning and its economics.

The 45-day path teaches this by building it. See the roadmap.

Start Week 1 free