Prompting changes what you ask. RAG (retrieval-augmented generation) changes what the AI can read when it answers, by looking up the right documents first. Fine-tuning changes the model itself with extra training on your examples. Start with prompting, add RAG when the model is missing facts, and consider fine-tuning only when you need the same style or format at large volume. The three can be combined.
- Prompting: clearer instructions and examples. Minutes of work, nothing to build. Always try it first.
- RAG: the AI searches your documents and answers from what it finds. Fixes “it does not know our stuff”.
- Fine-tuning: extra training on many examples of the job done right. Fixes “it is inconsistent”, not “it lacks facts”.
- Not either-or: OpenAI calls the approaches “additive, not exclusive”.
| Prompting | RAG | Fine-tuning | |
|---|---|---|---|
| What changes | Your instructions. | What the model reads at question time. | The model itself. |
| Fixes | Vague or badly framed requests. | Missing, private or recent facts. | Inconsistent format, tone or behaviour. |
| Effort | Minutes. | Hours to set up a search over your files. | Weeks, plus a set of good examples. |
| Update a fact | Edit the prompt. | Edit or remove the document. | Train again. |
| Example | “Reply in three sentences, friendly, no jargon.” | A help bot that checks this month’s policy before answering. | A model trained on hundreds of past replies to write in house style. |
The effort row follows this site’s prompt, context, skill or fine-tune ladder, and the examples are the site’s own; the rest is sourced below.
Picture an experienced new hire. Prompting is giving them a clear brief. RAG is handing them the company handbook so they can look things up before answering. Fine-tuning is weeks of training until they do the job your way without being told. You would not send someone on a training course just because they had not been given the handbook.
What is prompting?
The plain point: writing the request well, and checking that it works on more than one example.
OpenAI calls the careful version prompt engineering: “the process of writing effective instructions for a model, such that it consistently generates content that meets your requirements.”1 Its guide to accuracy puts it first: “Prompt engineering is typically the best place to start.”2
A prompt can include examples of good answers, the background the AI needs and the shape the reply should take. Nothing about the model changes; the next chat starts fresh. More on prompt meaning.
What is RAG?
The plain point: the AI looks things up first, then answers from what it found. The model stays the same.
Anthropic’s glossary: “In RAG, a language model is augmented with an external knowledge base or a set of documents that is passed into the context window.”3 The context window is everything the model can see while it writes one answer. OpenAI says RAG “is used to give the model access to domain-specific context to solve a task”, and that “many of our largest customer deployments at OpenAI were done using only prompt engineering and RAG”.2
The name comes from a 2020 paper by Patrick Lewis and colleagues, which joined a model’s built-in memory with a searchable index of Wikipedia.4 Pasting a document into a chat is the manual version; RAG automates the finding. More on RAG meaning.
What is fine-tuning?
The plain point: extra training that changes the model, so it does one job your way without long instructions.
Anthropic defines it as “the process of further training a pretrained language model using additional data.”5 OpenAI’s guide describes the practice: take a base model, “provide the kinds of inputs and outputs you expect in your application, and get a model that excels in the tasks you’ll use it for.”6 Its accuracy guide says it works by “showing it many examples of that task being performed correctly”.2 More on fine-tuning meaning.
How do you choose between them?
The plain point: ask whether the model lacks knowledge or lacks consistency. Knowledge is RAG; consistency may be fine-tuning; both start with a better prompt.
OpenAI’s accuracy guide uses exactly that split. “You need to optimize for context when 1) the model lacks contextual knowledge because it wasn’t in its training set.” And: “You need to optimize the LLM when 1) the model is producing inconsistent results with incorrect formatting.” It adds: “These two optimization methods are additive, not exclusive.”2 IBM sums up the difference: “RAG uses an organization’s internal data to augment prompt engineering, while fine-tuning retrains a model”.7
- The answer is vague or off-target: fix the prompt. Add the goal, the reader, an example and the format.
- It does not know your products, policies or this week’s figures: give it the documents, by pasting or through RAG.
- It knows enough but drifts in format or tone across thousands of requests: now fine-tuning is worth a look.
That checklist is this site’s reading of OpenAI’s split.
What can go wrong with each?
The plain point: none of them makes the AI reliably right; each has its own weak spot.
- Prompting stops at what the model knows. No wording supplies facts it never saw.
- RAG still guesses when the search finds nothing useful: a 2025 paper notes the way models are scored “still rewards guessing whenever search fails to yield a confident answer.”8 Hidden instructions in its documents can also reach the model; OWASP, a security project, says RAG and fine-tuning “do not fully mitigate prompt injection vulnerabilities.”9
- Fine-tuning is hard to undo. Remove a document from a RAG system and it is gone from the next answer; data trained into a model cannot be pulled back out that way.Reasoning, 13 Oct 2026 — as on this site’s AI terms people get wrong page: follows from the Anthropic glossary, which says retrieved documents are passed into the context window at question time, while fine-tuning is further training of the model. A fine-tune is also tied to the model it was built on, so when that model is retired the work may need redoing; see when AI models retire.
Climb in order: a better prompt, then the right documents, then fine-tuning. Before paying for training, ask whether the model is simply missing information; if it is, hand it the material.
Questions people ask
Is RAG better than fine-tuning?
For facts, usually yes: OpenAI points to context, not training, when the model lacks knowledge (source above). For a fixed format or tone at volume, fine-tuning can do what RAG cannot.
Can fine-tuning teach a model new facts?
It is the wrong tool for that. OpenAI’s guide sends missing knowledge to context and inconsistent behaviour to fine-tuning (source above). In this site’s reading, a model fine-tuned on facts can end up confidently wrong in your house style.
Can you use prompting, RAG and fine-tuning together?
Yes. OpenAI calls the methods “additive, not exclusive” (source above). A good prompt sits on top of whichever of the other two you use.
Is pasting a document into a chat the same as RAG?
It is the manual version of the same idea. RAG automates finding the right passage, which matters once there are thousands of documents; see RAG meaning.
Where to go next
- “Give me each word on its own.” Read prompt engineering, RAG and fine-tuning.
- “Where do skills fit?” See prompt, context, skill or fine-tune and what a skill in AI is.
- “How do I give the AI better material?” Read context engineering.
- “Help me write better prompts first.” Start with prompting 101.
- “How do I know a change helped?” See how to test a prompt.
QUICK CHECK · 3 QUESTIONS
Pick an answer to see if you got it. Nothing is recorded.
1. What does each approach change, according to the page?
- Prompting changes the model, RAG changes the prompt, fine-tuning changes the documents
- Prompting changes what you ask, RAG changes what the AI reads, fine-tuning changes the model
- All three change the model in slightly different ways
Show the answer
Answer: Prompting changes what you ask, RAG changes what the AI reads, fine-tuning changes the model. The page says prompting changes the request, RAG adds documents at question time, and fine-tuning retrains the model.
2. The AI does not know your company’s policies. What does the page suggest?
- Give it the documents, by pasting them or through RAG
- Fine-tune it on the policies straight away
- Repeat the question more firmly
Show the answer
Answer: Give it the documents, by pasting them or through RAG. The page says missing knowledge is a context problem, so the fix is handing the AI the material.
3. When does the page say fine-tuning is worth a look?
- Whenever a prompt gives one bad answer
- When the model needs facts from this week
- When the model drifts in format or tone across thousands of requests
Show the answer
Answer: When the model drifts in format or tone across thousands of requests. The page links fine-tuning to inconsistent behaviour at volume, not to missing facts.
- 1OpenAI, Prompt engineering, undated guide, read at source 1 Oct 2026 (as cited on this site’s prompt engineering meaning page): “Prompt engineering is the process of writing effective instructions for a model, such that it consistently generates content that meets your requirements.”↑
- 2OpenAI, Optimizing LLM Accuracy, undated guide, read at source 19 Oct 2026“Prompt engineering is typically the best place to start.” ↑aRAG “is used to give the model access to domain-specific context to solve a task”; “many of our largest customer deployments at OpenAI were done using only prompt engineering and RAG”. ↑bfine-tuning improves accuracy on a specific task by “showing it many examples of that task being performed correctly”. ↑c“You need to optimize for context when 1) the model lacks contextual knowledge because it wasn’t in its training set.” … “You need to optimize the LLM when 1) the model is producing inconsistent results with incorrect formatting.” … “These two optimization methods are additive, not exclusive.” ↑d
- 3Anthropic, Glossary, read at source 13 Oct 2026 (as cited on this site’s AI terms people get wrong page): “In RAG, a language model is augmented with an external knowledge base or a set of documents that is passed into the context window.”↑
- 4Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv:2005.11401, 22 May 2020, read at source 1 Oct 2026 (as cited on this site’s RAG meaning page): “the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever.”↑
- 5Anthropic, Glossary, read at source 13 Oct 2026 (as cited on this site’s AI terms people get wrong page): “Fine-tuning is the process of further training a pretrained language model using additional data.”↑
- 6OpenAI, Model optimization, undated guide, read at source 1 Oct 2026 (as cited on this site’s fine-tuning meaning page): “Fine-tuning lets you take an OpenAI base model, provide the kinds of inputs and outputs you expect in your application, and get a model that excels in the tasks you’ll use it for.”↑
- 7IBM, RAG vs. Fine-tuning, undated page, read at source 12 Oct 2026 (as cited on this site’s RAG meaning page): “RAG uses an organization’s internal data to augment prompt engineering, while fine-tuning retrains a model on a focused set”.↑
- 8Kalai et al., Why Language Models Hallucinate, arXiv:2509.04664, 4 Sep 2025, §5, read at source 11 Sep 2026 (as cited on this site’s RAG meaning page): “the binary grading system itself still rewards guessing whenever search fails to yield a confident answer.”↑
- 9OWASP, LLM01:2025 Prompt Injection, read at source 23 Sep 2026 (as cited on this site’s RAG meaning page): “While techniques like Retrieval Augmented Generation (RAG) and fine-tuning aim to make LLM outputs more relevant and accurate, research shows that they do not fully mitigate prompt injection vulnerabilities.”↑
Read at source 19 Oct 2026: OpenAI, Optimizing LLM Accuracy (undated).
Reused from this site’s meaning pages and articles, with the read dates recorded there: OpenAI’s prompt engineering and fine-tuning guides (prompt engineering and fine-tuning meaning, 1 Oct 2026); Anthropic’s glossary (AI terms people get wrong, 13 Oct 2026); the Lewis et al. paper, IBM’s RAG vs fine-tuning page, Kalai et al. and OWASP LLM01 (RAG meaning, 11 Sep–12 Oct 2026).
The comparison on this page (the new hire, the handbook and the training course), the table examples and effort row, and the advice marked “this site’s reading” are this site’s own, not claims from these sources.
Free printable AI cheat sheets, and AI news every week.
Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.
SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK