Yes, embeddings are computationally heavy, but they are not at all complicated and they provide a lot of benefit.

90% of "document" based RAG projects should view semantic search with embeddings as their primary method.

It's very powerful and so easy to implement that you could try it out and discover whether performance would be an issue rather than trying to anticipate it.

How is RAG any different from the search systems we've been building before LLMs? Is it the sudden need for everyone to design a search API and engine that's driven this trend?

If so, I'd like to see more design patterns around existing search problems:

- Correcting or backtracking based on feedback.

- Measuring relevance.

- Comparison with task-based pre-written queries. Does every LLM task need a full blown search engine? Why not a tightly scoped domain API for data retrieval?

So: https://en.wikipedia.org/wiki/Retrieval-augmented_generation

Can we not reward junk like this? Most of the sentences are incomprehensible and provide zero actual argumentation, it's just a list of "whats" with no "whys"

Oh boy...

Then, throw it all in BigQuery. Handles all the vector stuff natively.

Sprinkle an agentic bot UI thing on top to make it appear all-knowing and magical.

I assume other vendors than Google have a similar batteries-included approach you can just plug in.

Embeddings and semantic search add non determinism on top of non determinism. This seems fundamentally cursed. Lexical is much easier to control, iterate and debug. The tools are incredibly mature. Your users will probably prefer it as well.

https://github.com/jankovicsandras/plpgsql_bm25 BM25 search implemented in PL/pgSQL ( Unlicense / Public domain )

The repo includes also plpgsql_bm25rrf.sql : PL/pgSQL function for hybrid search ( plpgsql_bm25 + pgvector ) with Reciprocal Rank Fusion; and Jupyter notebook examples.