Many RAG systems handle in-document references (like “see appendix for details”) by building graphs or other preprocessing structures. The idea is to make sure cross-references are resolved before retrieval.
But with reasoning-based RAG, you don’t need that extra layer. The LLM itself can read the document, notice the reference, and then “jump” to the appendix (or wherever the reference points) to extract the answer. In other words, instead of pre-building structure, the model reasons its way through the content.
An example of reasoning-based RAG with PageIndex MCP is attached. In this example, the query asks for the total value. The main text only provides the increased value and refers to the appendix table for the total value. The LLM then looks up the appendix to find the total value and explains its reasoning process.
This raises an interesting question: how much preprocessing do we actually need for reasoning-augmented RAG, and when is it better to just let the model figure it out?
mingtianzhang•4mo ago
But with reasoning-based RAG, you don’t need that extra layer. The LLM itself can read the document, notice the reference, and then “jump” to the appendix (or wherever the reference points) to extract the answer. In other words, instead of pre-building structure, the model reasons its way through the content.
An example of reasoning-based RAG with PageIndex MCP is attached. In this example, the query asks for the total value. The main text only provides the increased value and refers to the appendix table for the total value. The LLM then looks up the appendix to find the total value and explains its reasoning process.
This raises an interesting question: how much preprocessing do we actually need for reasoning-augmented RAG, and when is it better to just let the model figure it out?