RAG, fine-tuning or prompting? A practical guide for business teams
Three ways to make a language model useful on your own data, and how to choose between them without a research team.
Use prompting when the model only needs your instructions. Use retrieval-augmented generation (RAG) when it must answer from private or changing documents with citations, which is the right default for most business assistants. Use fine-tuning only to teach a consistent style or narrow skill, never to add facts.
When teams decide to build an AI assistant on their own documents, three terms come up straight away: prompting, retrieval-augmented generation (RAG) and fine-tuning. They solve different problems, and picking the wrong one is the most common reason early AI projects disappoint.
What is prompting?
Careful instructions, examples and a clear output format will get surprisingly far. Prompting alone is enough when the model does not need your private knowledge: summarising, rewriting, classifying, extracting fields from text you pass in.
What is RAG (retrieval-augmented generation)?
RAG searches your documents for the passages relevant to a question and hands them to the model with the question. The model answers from that material and can cite it. This is the right default for “ask questions about our policies, contracts, tickets or wiki”.
- Knowledge stays in your systems and updates the moment a document changes.
- Answers can show their sources, so people can verify them.
- No model training is required, which keeps cost and risk low.
What is fine-tuning, and when is it worth it?
Fine-tuning adjusts a model using examples so it adopts a style, format or narrow skill consistently. It is not a good way to add facts, because those go stale and cannot be cited. Consider it when you have hundreds of high-quality examples and a stable task, such as a house tone of voice or a specialised classification.
How do you choose between them?
| You need… | Start with |
|---|---|
| Better wording, structure or classification of text you supply | Prompting |
| Answers from private or changing documents, with sources | RAG |
| A consistent style or niche skill, with many good examples | Fine-tuning |
| All of the above | Prompting + RAG first, fine-tune only if a gap remains |
Whichever you choose, measure it
Build a test set of real questions with known good answers and score every change against it. Without that, you are guessing. See our evaluation checklist for how to set one up.