7 min read

GraphRAG

Ask a vector RAG system "what are the main complaints customers raised this quarter?" and it retrieves the five chunks most similar to that sentence, which are probably five complaints. Not the main themes across forty thousand tickets. The question is about the whole collection, and similarity search can only ever return a few pieces of it.

GraphRAG adds structure that vectors do not have: the entities in your documents (people, products, places, policies), the relationships between them, and summaries of clusters of related things. Microsoft Research published the best known version in 2024, and the idea has spread into many tools since. It is powerful for a specific kind of question, and expensive enough that you should know exactly when you need it.

Your answer

Why can't a vector search over chunks answer 'what are the five biggest themes across all our support tickets this quarter?', no matter how good the embeddings are?

How the index is built#

1/4 A model reads every chunk and pulls out entities ("Pro plan", "UPI", "Bengaluru", "rider app") and relationships between them ("UPI refund delayed for Pro plan users"). This is one model call per chunk, and it is where most of the cost goes.