A Picture of RAG: Retrieval-Augmented Generation Explained Visually

A RAG pipeline becomes much easier to follow once we can visualize the path of information flowing through it. First, a user question triggers retrieval from an external source, then selected evidence is added to the model's input, and, finally, the LLM generates a response with that evidence available in context.

But what all happens in between query and retrieval? This visual explainer focuses on how information is prepared, retrieved, added to the request, and passed to the model.

The RAG Image — The Pipeline in One Picture

In the basic retrieve → augment → generate sequence, retrieval finds external information relevant to the question, augmentation adds selected evidence to the model input, and generation produces a response using the question and retrieved context together.

The basic RAG flow: retrieve evidence, add it to the model input, then generate an answer.

In this sequence, retrieval selects the external evidence available for the request, and the LLM interprets the supplied context and generates the response.

The Preparation of Knowledge

Before any questions arrive, document-based RAG pipelines usually process their source material into a searchable knowledge base.

Documents are often divided into smaller retrievable units called chunks. Their boundaries affect what search can return later, which makes chunking strategy an important part of retrieval design.

In vector-based RAG, an embedding model converts each chunk into a numerical representation of its semantic content. Those embeddings can be stored and searched in a vector database or another vector-capable index.

Then, when a question is posed to the system, the query is encoded into the same embedding space. Similarity search compares that query vector with the indexed vectors and retrieves the associated passages ranked as relevant to the request.

A common RAG pipeline prepares searchable knowledge first, then retrieves from it when a question arrives.

Retrieval can also use lexical search, SQL databases, APIs, or graph-based knowledge, depending on the source and the question.

What Does RAG Actually Retrieve?

A typical customer question such as:

"When can I cancel the Pro plan?"

Prompts a chunk-based retriever to search the indexed passages and rank the ones most relevant to that question, with the retrieved output serving as source material for later generation.

A product knowledge source might contain hundreds or thousands of passages, with the search returning a few results like:

• Chunk 14: Pro subscriptions can be cancelled at any time.
• Chunk 27: Cancellation takes effect at the end of the current billing period.

Retrieval selects passages related to the query instead of sending the entire knowledge source to the model.

Thousands of passages can exist in the source while only a small subset enters the final model context. Information that exists in the corpus can consequently be absent from the answer if retrieval never selects it or the application doesn't include it in the request.

What the LLM Sees With RAG

Once retrieval has found the relevant passages, the application assembles the model input.

A request can already contain system instructions, the user's question, and conversation history. RAG adds retrieved material that the model can reference during generation. For the cancellation example, that context could include the question alongside the two policy passages above and an instruction to answer using the supplied evidence.

RAG adds retrieved evidence to the information the LLM receives for generation.

The retrieved information becomes part of the model's contextual memory for that inference. The application can supply different context on the next request without changing the model parameters, which lets a general-purpose model answer from company documentation, recent records, permissioned private data, or other information available through retrieval.

The model doesn't permanently learn those passages just because they appeared in a RAG request. Information needed again later has to be retrieved again, supplied through reconstructed context, or preserved in an external memory layer like cognee.

Retrieved knowledge changes the information available at inference time without retraining the model.

Updating the source data or its retrieval index can change what the model receives on a later request without requiring another model-training process.

Retrieval Quality Affects What Reaches the LLM

For the cancellation question, retrieving the cancellation policy and billing passage gives the model evidence that directly addresses the request. A search that returns unrelated pricing material leaves generation without the information needed for a reliable answer.

Retrieval quality changes the evidence available to the model before generation begins.

Good retrieval still doesn't guarantee a correct response. The source can be incomplete or outdated, and the LLM can misread accurate material or generate claims that go beyond it. Grounding generation in retrievable sources can reduce unsupported answers, but production systems still need safeguards for LLM hallucinations and retrieval failures.

Production RAG can incorporate metadata filters, hybrid search, reranking, graph traversal, or evaluation into the pipeline, while the basic relationship stays the same: retrieval controls which external information becomes available to generation, and the LLM still has to use that information correctly.