What Is Retrieval-Augmented Generation, and Why Does It Exist?

Three problems a language model cannot fix by itself

A large language model's knowledge is frozen at the moment its training data was collected. Ask it about an event from last week, or the contents of a document it has never seen, and it has two options: say it does not know, or produce a fluent, confident, wrong answer. The second option is hallucination, and it is not a bug that better training eventually removes, it is a structural consequence of a model that only knows what was in its training set.

Three specific gaps follow from this: the model's knowledge is stale (nothing after its training cutoff), closed (nothing that was never public, like your company's internal wiki or a customer's private records), and unverifiable (an answer with no way to check where it came from). Fine-tuning the model on new data helps with staleness, but it is slow, expensive, and still gives you an answer with no citation trail.

Retrieval as the fix

Retrieval-augmented generation (RAG) takes a different approach: instead of trying to bake all knowledge into the model's weights, it fetches relevant information at the moment of the question and hands it to the model as context. The model's job changes from "recall the answer from memory" to "read this material and answer based on it," the same shift a student makes between a closed-book exam and an open-book one.

This single idea, retrieve first, then generate, is the seed every architecture in this course grows from. What differs from one architecture to the next is how the retrieval step decides what to fetch, how many times it fetches, and what it does when the first fetch is not good enough. Lesson 2 lays out the three-stage pipeline shared by all of them before this course starts telling the architectures apart.