I could envision a bunch of use cases about this workikg well. Ive personally encounter scenadios where sometimes the ai gets hung up on irrelevant outdated fact. But could still look up if specifically needed.

I could see even an automated short summary of all history that is outdated being updated in the vector db from this too. So not all context is lost.

Keep up the great work!

No shade on your project, this is an emerging space and we can all use novel approaches.

Keep it up!

The use of commit-hooks is also very clever (mentioned here in the replies)

For code search only, BM25 might be a bit overkill and not exactly what you want. FM indexes would be a simpler and faster way to implement pure substring search.

Maybe having both kinds of search at the same time could work better than either in isolation. You could frame them as "semantic" and "exact" search from the perspective of the LLM tool calls. The prompt could then say things like "for searching the codebase use FunctionA, for searching requirements or issues, use FunctionB."

That is what's known in FAISS as a "flat" index, just one thing after another. And obviously you can query by primary key to the key-value store that is git, and do atomic updates as you'd expect. In SQL land this is an unindexed column, you can do primary key lookups on the table, or you can look through every row in order to find what you want.

If you don't need fast query times, this could work great! You could also use SQL (maybe an AWS Aurora Postgres/MySQL table?) and stuff the fact and its embedding into a table, and get declarative relational queries (find me the closest 10 statements users A-J have made to embedding [0.1, 0.2, -0.1, ...] within the past day). Lots of SQL databases are getting embedding search (Postgres, sqlite, and more) so that will allow your embedding search to happen in a few milliseconds instead of a few seconds.

It could be worth sketching out how to use SQLite for your application, instead of using files on disk: SQLite was designed to be a better alternative to opening a file (what happens if power goes out while you are writing a file? what happens if you want to update two people's records, and not get caught mid-update by another web app process?) and is very well supported by many language ecosystems.

Then, to take full advantage of vector embedding engines: what happens if my embedding is 1024 dimensions and each one is a 32 bit floating point value? Do I need to save all of that precision? Is 16-bit okay? 8-bit floats? What about reducing the dimensionality? Is it good enough accuracy and recall if I represent each dimension with an index to a palette of the best 256 floats for that dimension? What about representing each pair of dimensions with an index to a palette of the best 256 pairs of floats for those two dimensions? What about, instead of looking through every embedding one by one, we know that people talk about one of three different topics, and we have three different indices for each of those major topics, and to find your nearest neighbors you want to first find your closest topic (or maybe closest two topics?) and then search in those lower indices? Each of these hypotheticals is literally a different “index string” in an embedding search called FAISS, and could easily be thousands of lines of code if you did it yourself.

It’s definitely a good learning experience to implement your own embedding database atop git! Especially if you run it in production! 100MB is small enough that anything reasonable is going to be fast.

Also, there are tradeoffs associated with using BM25 instead of embedding similarity. You're essentially trading semantic understanding for computational speed and keyword matching.

[0] https://github.com/xhluca/bm25s