GlossaryUpdated
RAG (retrieval-augmented generation)
RAG in AI means retrieval-augmented generation: before the AI answers, it looks up the most relevant passages in your documents or on the web and writes its answer from them.
RAG in AI stands for retrieval-augmented generation: before the AI writes an answer, it looks up the most relevant passages in your documents, a company knowledge base or the web, and writes the answer from what it found. Retrieval is the lookup. Generation is the writing. The point is simple: an AI model only knows what was in its training data, which stops at a date and never included your company's files. RAG hands it the right pages at the moment you ask.
If you have ever uploaded a PDF to ChatGPT and asked a question about it, or seen a list of sources under an AI answer, you have used RAG. You did not need to know the name.
What it means for you at work
RAG is the reason AI tools can answer questions about your own work at all. Three things follow from how it works, and each one changes how you should use those tools.
- The AI only reads a few passages, not your whole folder. It searches first, keeps the handful of chunks that look most relevant, and answers from those. If the answer is in a document it did not pick, it will not see it. A question that names the document, the client or the date helps it pick the right one.
- The answer is only as good as the files. If your shared drive holds three versions of the pricing sheet, the AI may quote the old one. Clean sources beat clever prompts.
- Sources are your check. A RAG answer can point to where each claim came from. Use that. A claim with no source, or a source that does not say what the answer says, is the first place to look for a mistake.
How RAG works, step by step
Behind the chat box, a RAG system does four things every time you ask:
- Split. Ahead of time, your documents are cut into short pieces, often a few paragraphs each. The pieces are called chunks.
- Index. Each chunk is stored in a way that lets the system find it by meaning, not only by exact words. A question about "holiday allowance" can find a chunk that says "annual leave".
- Retrieve. When you ask a question, the system finds the chunks that match it best and keeps the top few.
- Generate. The AI gets your question plus those chunks, with an instruction to answer from them. Good tools also tell it to name which chunk each claim came from.
Step 3 is where most problems start. If the search keeps the wrong chunks, the AI writes a confident answer from the wrong material.
Where you already use RAG
You meet RAG in most AI tools at work, under other names:
- File uploads and projects. Asking ChatGPT, Claude or Gemini about a PDF, a spreadsheet export or a set of files saved in a project.
- Answers from your work apps. Microsoft 365 Copilot answering from your email, Teams chats and SharePoint files, or Gemini answering from Gmail and Drive. Many of these connections run through MCP: a connector that lets AI use your apps, such as Drive, Slack or Notion. See MCP.
- Notebook tools. Google's NotebookLM answers only from the sources you add and shows a citation for each claim.
- Search with AI answers. Perplexity, ChatGPT search and Google's AI Overviews look up web pages first and summarise them, with links.
- Company help bots. A support chatbot that answers from a company's help centre is usually a RAG system over those articles.
Agentic RAG: when the AI decides what to look up
Plain RAG does one search and one answer. Agentic RAG is RAG run by an agent: AI that completes a task with several steps for you, not just one answer. The AI plans the lookup itself.
In practice that means the AI:
- Breaks your question into smaller questions.
- Searches for each one, sometimes in different places (your files, the web, a database).
- Reads what came back and decides whether it is enough.
- Searches again with better words if it is not.
- Writes the answer only when it has what it needs, with sources.
The deep research features in ChatGPT and Gemini are the version most people have tried: you ask one question, and the tool spends several minutes running many searches before it writes a report. The gain is coverage. A one-shot search misses whatever your first wording did not match; an agent that searches again catches more. The cost is time, and more places for a wrong turn. An agent that picks a weak source early can build the whole report on it.
Why RAG answers still go wrong
RAG cuts down on made-up answers, but it does not remove them. The common failures are easy to spot once you know them:
- Wrong chunk. The search found something close but not right, such as last year's policy instead of this year's.
- Missing chunk. The answer is in a table or a scanned page the system could not read, so the AI answers without it.
- Filling the gap. The sources cover half the question, and the AI fills the other half from its own memory without saying so.
- Instructions hidden in a document. A web page or file can contain text written to steer the AI. Because RAG feeds documents straight into the AI, this is a real risk. See prompt injection.
The fix in every case is the same habit: ask the AI to show where each claim came from, and open the source for anything that matters.
RAG vs a bigger context window vs fine-tuning
There are three ways to get your information into an AI's answer:
- Paste it in. Newer models can read very long inputs, so for one report or a few files, pasting or uploading the whole thing works and nothing gets missed. This is part of context engineering: choosing what the AI reads before it answers.
- RAG. For a whole drive, a help centre or years of email, there is too much to paste. RAG searches and passes in only the relevant parts.
- Fine-tuning. Training the model further on your examples changes how it writes or behaves. It is a poor way to teach it facts that change, because every change means training again.
For most people at work the rule is: paste it when it fits, let the tool search when it does not.
A prompt that makes RAG answers checkable
Use this when you ask an AI about uploaded files or connected documents:
Answer my question using only the documents I have shared or connected.
<question>[your question]</question>
Rules:
1. After each claim, quote the exact sentence it comes from and name the document (and page or section if you can see it).
2. If the documents do not answer part of the question, say "not in the documents" for that part. Do not fill it in from general knowledge.
3. If two documents disagree, show both and say which one looks newer.
End with a list of the documents you used and any you searched but did not use.
Rule 2 is the one that matters most. Without it, the AI quietly fills gaps from memory, and you cannot tell which parts came from your files.
RAG (retrieval-augmented generation) in the news
- JEV's LLM Retrieval Beats Embeddings by 10x: What It Means for Your RAG Pipeline
Oct 13, 2026
- AIUC Raises $40M to Make Frontier AI Models Auditable by Default
Sep 16, 2026
- The Agentic Memory Stack Is Fracturing: What Builders Need to Know Now
Jul 15, 2026
- The Reversal Curse: Why Your LLM Knows A→B but Not B→A
Jun 23, 2026
- Qwen 27B's Full 256K Context Now Fits on a 24 GB Mac
Nov 5, 2026
- OntoPrune Cuts 85% of LLM Context Tokens and Speeds CPU Inference 6.7x
Oct 5, 2026
- Secrets Extraction Against Black-Box LLMs: What Attackers Are Actually Doing
Oct 1, 2026
- The AI Agent Tooling Layer Is Filling In Fast
Aug 25, 2026
Questions
What does RAG stand for in AI?
In AI, RAG stands for retrieval-augmented generation. Retrieval means looking up information. Generation means writing the answer. Augmented means the lookup adds to what the AI already knows. Put together: the AI finds the relevant parts of your documents or the web first, then writes its answer from them.
What is agentic RAG?
Agentic RAG is RAG run by an agent: AI that completes a task with several steps for you, not just one answer. Instead of one search and one answer, the AI decides what to search for, reads the results, judges whether they are enough, searches again if not, and only then writes. Deep research features in ChatGPT and Gemini work this way.
Is ChatGPT a RAG system?
Not on its own. The model answers from what it learned in training. It becomes RAG when it looks something up first: when it searches the web, reads a file you uploaded, or searches a connected app before answering. When you see sources or citations under an answer, some form of RAG was used.
Does RAG stop AI from making things up?
It helps, but it does not stop it. The AI can still pick the wrong passage, miss the right one, or add details the source never said. Ask it to quote the exact sentence it relied on, and open the source for anything you will repeat to others.
What is the difference between RAG and fine-tuning?
Fine-tuning changes the model itself by training it further on examples, which takes time and money and is usually done by a technical team. RAG leaves the model alone and hands it the right documents at the moment you ask. For keeping up with documents that change every week, RAG is the usual choice.
Who invented RAG?
The name comes from a 2020 research paper by Patrick Lewis and colleagues at Facebook AI Research (now Meta), University College London and New York University, titled Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. The idea of searching first and answering second is older, but that paper gave it the name everyone now uses.
Related terms
- Context engineeringChoosing what you give AI to read, not only how you ask.
- MCPA connector that lets AI use your apps, such as Drive, Slack or Notion.
- Prompt injectionHidden text in a page or email that tricks AI into following the wrong orders.
- Agent memoryWhat AI saves so it remembers you and your work next time.