Limitations of Naive RAG

Retrieval-Augmented Generation (RAG) in its basic form — embed a query, fetch the top-K most similar chunks, feed them to the model — has well-known failure modes that surface once you move past simple Q&A.

Multi-page answers. When the answer to a question is spread across several documents, top-K retrieval may only surface some of the relevant chunks. The model gets a partial picture and either gives an incomplete answer or hallucinates the missing parts.

Exact match misses. If a user asks for a specific function signature, config flag, or error message, the embedding similarity search might not rank the exact chunk highly enough. Embeddings capture semantic similarity, not lexical precision, so the chunk containing the literal string can lose to a more "semantically similar" but less useful result.

No browsing. Basic RAG is a single-shot retrieve-then-generate pipeline. The model can't follow up — it can't say "that chunk mentions a related page, let me look at that too." There's no exploration or iteration.

These limitations push teams toward giving agents richer access to the underlying data. Mintlify addressed this by building a virtual filesystem (ChromaFs) that lets agents browse documentation with shell commands instead of relying on chunk retrieval. Other approaches include recursive retrieval (using an initial retrieval to refine the query), graph-based RAG (following entity relationships between chunks), and agentic RAG (letting the model decide when and how to retrieve).

The common thread is that retrieval needs to become interactive rather than one-shot — the agent should be able to explore, not just query.