Agentic RAG: Planning, Tool Use, and Iterative Retrieval

Retrieval as a decision, not a fixed step

Every architecture so far treats retrieval as something that always happens, once, as a mandatory step in a pipeline. Agentic RAG puts an LLM-based agent in control of that decision instead: the agent decides whether to retrieve at all, what to search for, which tool or data source to use, and whether the results so far are enough or another retrieval pass is needed.

What "agentic" adds concretely

Three capabilities distinguish agentic RAG from everything earlier in this course. Planning: the agent can break a complex question into sub-questions and retrieve for each one separately, rather than firing one query at the whole question. Tool use: retrieval is not limited to one document store, the agent can choose between a vector search, a SQL query, a web search, or a calculator, depending on what the sub-question actually needs. Iteration: after seeing a retrieval result, the agent evaluates whether it actually answers the current sub-question, and if not, retrieves again with a refined query, rather than being limited to one shot.

The cost of that flexibility

Agentic RAG trades predictability and latency for capability. A naive RAG call is one retrieval and one generation, fast and consistent. An agentic RAG call might make anywhere from one to a dozen tool calls depending on the question, which means variable latency, variable cost, and a real risk of the agent looping or retrieving unhelpfully without a hard stop condition. This is exactly the tradeoff Adaptive RAG (Lesson 20) is built to manage: use this expensive flexibility only on the questions that actually need it.