I’ve been working on a project called imesde (In-MEmory Streaming Data Engine). It’s a Rust-based vector engine designed specifically for data that doesn't need to live forever—like logs, financial tickers, or live chat context.
Most vector DBs are built for persistence and massive indexing (HNSW/Disk). I’m exploring a different trade-off: zero-disk dependency and circular buffers.
> What I’m trying to solve:
Latency Tax: By staying CPU-first (optimized for Apple Silicon/NEON) and using in-process ONNX, I’m seeing sub-300μs P99 latency for small, high-frequency streams.
Instant Anomalies: I implemented a moving centroid update, so you can detect "semantic outliers" during ingestion without extra passes.
It’s still in the early validation phase and very much an experiment. I’d love to know if you think this "ephemeral" approach makes sense for real-time RAG or if I’m over-optimizing a niche.
alessiopel•1mo ago
Most vector DBs are built for persistence and massive indexing (HNSW/Disk). I’m exploring a different trade-off: zero-disk dependency and circular buffers.
> What I’m trying to solve:
Latency Tax: By staying CPU-first (optimized for Apple Silicon/NEON) and using in-process ONNX, I’m seeing sub-300μs P99 latency for small, high-frequency streams.
Instant Anomalies: I implemented a moving centroid update, so you can detect "semantic outliers" during ingestion without extra passes.
It’s still in the early validation phase and very much an experiment. I’d love to know if you think this "ephemeral" approach makes sense for real-time RAG or if I’m over-optimizing a niche.
Repo: https://github.com/imesde/imesde
Feedback (even the "this is a bad idea" kind) is very welcome.