8 min read

Agentic RAG

Classic RAG retrieves once, then answers. That is fine for "what is the refund window?". It breaks on "which of our three pricing plans would have been cheapest for this customer last quarter, given their usage?": the first search reveals what to search next, and nothing in a single shot pipeline can act on that.

Agentic RAG turns retrieval into a tool. The model decides whether to search, what to search for, reads the results, and searches again until it has enough, or decides the answer is not there. It is the agent loop with search as the main tool, and it is how deep research features and coding assistants find things.

Your answer

Your one-shot RAG system gets multi-step questions wrong. What would the model need to be able to do, that the fixed pipeline does not let it do?

From pipeline to loop#

Shape

Code decides: always retrieve, always once, always answer. Fast, cheap and predictable, and it has no way to recover when the first search misses or the question needs several hops.

A multi-hop question, step by step#

1/7 First the model needs to know what the plans are. It writes the query itself.

Patterns with names#

Retrieval as a tool. The base pattern above: give the model a search tool with a good description, and let it decide. Most agent frameworks make this a few lines.

Self-checking (Self-RAG, Corrective RAG). Research from 2023 and 2024 showed that having the model judge the retrieved chunks before answering helps: if they look irrelevant, rewrite the query and search again, or fall back to a different source such as web search. In practice this is a grading step in the loop.

Router over sources. Several retrieval tools (help docs, product catalogue, SQL, web), each with a sharp description, and the model picks per question.

What it looks like in code#

The heart of agentic RAG is just a well described search tool handed to an agent loop. Here it is with the Anthropic SDK's tool runner, which runs the loop for you.

python

import anthropic
from anthropic import beta_tool
 
client = anthropic.Anthropic()
 
@beta_tool
def search_docs(query: str, product: str = "any") -> str:
    """Search the help centre. Use focused queries of a few words.
    Call again with a different query if results look unrelated.
 
    Args:
        query: What to search for, e.g. "Team plan seat minimum".
        product: Limit to one product, or "any".
    """
    hits = retriever.search(query, filters={"product": product}, k=5)
    return "\n\n".join(f"[{h.id}] {h.title}\n{h.text}" for h in hits)
 
runner = client.beta.messages.tool_runner(
    model="claude-opus-5-5",
    max_tokens=16000,
    system="Answer only from search results. Cite ids like [doc_12]. "
           "If the results do not contain the answer, say so.",
    tools=[search_docs],
    messages=[{"role": "user", "content": question}],
)
for message in runner:  # one message per turn of the loop
    final = message

retriever stands for whatever search you already have: hybrid search with reranking from the earlier chapters. Agentic RAG does not replace good retrieval. It calls it more cleverly.

Choice	What you gain	What you pay	Pick it when
Classic, one-shot RAG	One retrieval, one call: fast, cheap, easy to evaluate.	No recovery from a bad search, no multi-hop questions.	FAQ style questions answered by one passage. Most support traffic.
Agentic RAG	Multi-hop questions, self-correction, several sources.	Several model calls per question, variable latency, harder to evaluate.	Research, analysis, and questions whose next step depends on what was found.
Route first, then decide	Simple questions stay cheap, hard ones get the loop.	A classifier to build and evaluate.	Mixed traffic. Usually the right production answer.

One shot or agentic?

"What is the return window for shoes?"

"Which of my last five orders were eligible for a late refund?"

"Summarise what changed in our pricing this year"

A latency budget of 800 ms

Give the loop a budget

A model that keeps searching because nothing looks quite right will happily run thirty searches. Cap the tool calls, tell the model in the system prompt it may say the answer is not available, and log every search so you can see when it is flailing. The loop engineering chapter covers the rest.

Checkpoint#

Checkpoint

1. What is the core change from classic RAG to agentic RAG?

2. What do Self-RAG and Corrective RAG add?

3. Your support bot has a 1 second latency target and mostly FAQ questions. Agentic or classic?

Worth memorising

Agentic RAG = search as a tool inside an agent loop.
Use it for multi-hop and research questions, not FAQs.
Self-check retrieved results, re-search or switch source on a miss.
Budget the loop and allow "the answer is not available".

Say this in 60 seconds

Classic RAG retrieves once and answers, which cannot handle questions where the first result decides the next search. Agentic RAG makes retrieval a tool inside an agent loop: the model chooses whether to search, which source, what query, and when it has enough, and it can check its own retrieved results and re-search when they miss, which is the idea behind Self-RAG and Corrective RAG. It costs several model calls per question, so I route simple questions to one-shot RAG and send only multi-hop or research questions into the loop, with a cap on searches and explicit permission to say the answer is not available.

Next: GraphRAG Related: What an agent is