I completely agree with the insight that full text search has been complexified. People seem to want to jump straight to clustering or other enterprise level things.
I also appreciate the moxie of getting in there and building it yourself.
Myself, I reach for Lucene. Then you don’t need to build all this yourself if you don’t want. It lives in a dir on disk. True, it’s a separate database, but one optimized for this problem.
aorloff•1h ago
This was the solution I was thinking about, but I thought, well that's the way someone would have done it 20 years ago
mobeigi•1h ago
Great read. It makes you wonder how heavily optimised the tokenizers used by popular search enginea truly are.
eduction•1h ago
I also appreciate the moxie of getting in there and building it yourself.
Myself, I reach for Lucene. Then you don’t need to build all this yourself if you don’t want. It lives in a dir on disk. True, it’s a separate database, but one optimized for this problem.
aorloff•1h ago