Hello HN! I've been having this problem with VertexAI's RAG for some time now, basically if you ask Gemini (with RAG) for something and it doesn't have the answer, then it simply says it can't find it, without doing fallback to the general model.

So, in documenting myself I came across LLM-as-a-critique (also LLM-as-judge), a methodology whereby a second LLM is used to evaluate the answers of the first one.

This gave rise to Kritikos, a simple and lightweight backend written in Go that allows you to take advantage of this methodology. I invite you to leave a comment if you too have encountered this problem, I am of course open to improvements as well, let me know what you think!