← All lab updates
RAG, explained: how an assistant answers from your documents, and the four places it goes wrong
Most company assistants answer from the company's own documents through a method called RAG, retrieval-augmented generation. It works by finding the passages closest to the question and handing them to the model. Here is each step in plain words, five lines of code, and the four places where it quietly goes wrong.
## The idea
A language model only knows what it saw in training. To answer about your products, contracts or policies, it needs your documents in view at the moment it answers. **RAG** (retrieval-augmented generation) does exactly that: it finds the passages most related to the question and puts them in front of the model.
## The steps
1. **Cut.** Your documents are split into chunks of a few hundred words.
2. **Embed.** Each chunk becomes a vector, a list of numbers where texts with similar meaning sit close together (the same idea we saw inside the model in [how attention works](/news/tutorials/how-a-transformer-reads-attention)).
3. **Search.** The question is turned into a vector too, and the closest chunks are fetched.
4. **Answer.** The model receives the question plus those chunks and writes the answer.
```python
chunks = split(documents, size=500) # cut the documents into pieces
index = [embed(c) for c in chunks] # each piece becomes a vector
q = embed(question) # so does the question
top = nearest(q, index, k=5) # the 5 closest pieces
answer = model(question + "\n" + join(top)) # the model answers with them in view
```
## Four places it goes wrong
1. **The cut.** A chunk can separate a rule from its exception. The model sees "refunds are allowed" without the next paragraph that says "except after 30 days".
2. **Closeness is not truth.** The search finds what sounds similar, not what is correct or current. An old version of a policy can be the closest match.
3. **The question steers the search.** A question that embeds its own conclusion ("courts have firmly established that...") pulls back the documents that agree with it. We wrote about this in [the framed question](/news/security/framed-question-steers-retrieval).
4. **The answer hides its sources.** A model can state retrieved facts as bare assertions, without saying where they came from. From the text alone, a grounded answer and an invented one can look identical.
## What to check in your own assistant
- Ask the same question in neutral and in loaded wording: do different documents come back?
- Look at which chunks were used, not only at the answer.
- Ask questions your documents cannot answer: does it say so, or does it invent?
## Go further
- [RAG and retrieval](https://xbe.at/llms-and-genai/topic/rag-and-retrieval/), 39 practical articles on xbe.at
- [Prompt, retrieval or fine-tune](/news/tutorials/prompt-retrieval-or-fine-tune): when RAG is the right fix
- Our product for this: [catching a biased search](/products#search)
This update is not available.