I've been benchmarking Byterover CLI's Context Tree against standard Vector RAG, specifically using Gemini Flash on a ~1,300 file codebase.
The consensus is usually that Gemini is chaotic or lazy compared to Claude. My theory was that "context stuffing" was the main cause, causing context dilution.
Byterover moves the retrieval logic client-side: it builds a dependency tree locally (AST-based) and prunes irrelevant files before sending the prompt. This contrasts with Vector RAG, which often retrieves the wrong Auth implementation files in my tests.
The Result: With the Context Tree cleaning the window, Gemini Flash stopped hallucinating imports and outperformed Claude 3.5 Sonnet on the "Refactor Auth Controller" task.
The post includes the full benchmarks and reproduction steps.
lanchiii•4w ago
The consensus is usually that Gemini is chaotic or lazy compared to Claude. My theory was that "context stuffing" was the main cause, causing context dilution.
Byterover moves the retrieval logic client-side: it builds a dependency tree locally (AST-based) and prunes irrelevant files before sending the prompt. This contrasts with Vector RAG, which often retrieves the wrong Auth implementation files in my tests.
The Result: With the Context Tree cleaning the window, Gemini Flash stopped hallucinating imports and outperformed Claude 3.5 Sonnet on the "Refactor Auth Controller" task.
The post includes the full benchmarks and reproduction steps.