Hi HN, I am Anil and I am building Rice (https://tryrice.com), a low latency context orchestration layer for AI agents.
Rice replaces the standard HNSW vector search with Weightless Neural Networks (WNNs) to enable O(1) retrieval speeds, specifically designed for realtime voice agents and high-frequency multi agent workflows.
The problem we ran into while building voice agents was simple: Latency kills immersion.
Between STT (Speech-to-Text), the LLM inference, and TTS (Text-to-Speech), we had a strict latency budget. Spending 200ms+ on a Vector DB lookup (plus reranking) was eating up too much of that budget. On top of that, we found that stateless RAG meant our agents were constantly hallucinating permissions and accessing data they shouldn't, or failing to remember a constraint set by another agent 10 seconds ago.
The industry standard is to throw everything into Pinecone or pgvector and handle the logic in the application layer. That works for chatbots, but for autonomous agents that need mutable memory (read/write state 50 times a minute), standard vector indexes are too heavy and slow to update.
Rice is our attempt to fix the Working Memory problem.
Under the hood:
Rice is an indexing and state management engine that sits between your LLM and your data.
Instead of using HNSW graphs (which are O(log N)), we rely on Weightless Neural Networks (similar to WiSARD architectures).
- Deep Semantic Hashing: We train a lightweight model to compress dense embeddings into sparse binary codes while preserving semantic relationships.
- O(1) Lookup: These binary codes are mapped directly to memory addresses. This effectively turns "Search" into a hash table lookup.
The Result: Retrieval latency stays flat (<50ms) even as your context grows to millions of items, and updates to the memory state are instant (no reindexing penalty).
We wrap this WNN core in a State Machine that handles Access Control (ACLs). When an Agent requests context, Rice checks the identity and state before the retrieval, ensuring you don't leak data between users or agents. Think of it as "Supabase for Agent Context", a managed backend that handles the memory graph and security policies so you don't have to write raw SQL RLS queries for every RAG call.
Where we are now
Rice is currently in closed beta/alpha. We are working with a few design partners in the voice and support automation space who need that sub 100ms retrieval speed.
We know using WNNs for semantic search is a contrarian bet compared to the massive investment in Vector DBs. We are specifically optimizing for "Hot State" (short term, high velocity memory) rather than "Cold Storage" (archival knowledge), though the lines are blurring.
Use Cases we are seeing:
- Voice Agents: Shaving 200ms off RAG latency to make conversation feel natural.
- Multi-Agent Hand-offs: Agent A (Sales) updates a "Customer Mood" state, and Agent B (Support) sees it instantly without hallucinating.
- Internal Tools: Enforcing strict ACLs (e.g., "Junior Devs can't query the Salary Table") at the infrastructure layer.
We are looking for engineers who are pushing the limits of agent latency or struggling with state management to try it out and tell us where it breaks.
I’m especially interested in hearing your skepticism on the WNN approach - we know it’s weird, but for our specific constraints, the speed tradeoff has been worth it.
ob_mobly•1d ago
Interesting take on the matter. Joined the waitlist, would like to see it in action.
aperi•1d ago
Rice replaces the standard HNSW vector search with Weightless Neural Networks (WNNs) to enable O(1) retrieval speeds, specifically designed for realtime voice agents and high-frequency multi agent workflows.
The problem we ran into while building voice agents was simple: Latency kills immersion.
Between STT (Speech-to-Text), the LLM inference, and TTS (Text-to-Speech), we had a strict latency budget. Spending 200ms+ on a Vector DB lookup (plus reranking) was eating up too much of that budget. On top of that, we found that stateless RAG meant our agents were constantly hallucinating permissions and accessing data they shouldn't, or failing to remember a constraint set by another agent 10 seconds ago.
The industry standard is to throw everything into Pinecone or pgvector and handle the logic in the application layer. That works for chatbots, but for autonomous agents that need mutable memory (read/write state 50 times a minute), standard vector indexes are too heavy and slow to update.
Rice is our attempt to fix the Working Memory problem.
Under the hood:
Rice is an indexing and state management engine that sits between your LLM and your data. Instead of using HNSW graphs (which are O(log N)), we rely on Weightless Neural Networks (similar to WiSARD architectures).
- Deep Semantic Hashing: We train a lightweight model to compress dense embeddings into sparse binary codes while preserving semantic relationships. - O(1) Lookup: These binary codes are mapped directly to memory addresses. This effectively turns "Search" into a hash table lookup.
The Result: Retrieval latency stays flat (<50ms) even as your context grows to millions of items, and updates to the memory state are instant (no reindexing penalty).
We wrap this WNN core in a State Machine that handles Access Control (ACLs). When an Agent requests context, Rice checks the identity and state before the retrieval, ensuring you don't leak data between users or agents. Think of it as "Supabase for Agent Context", a managed backend that handles the memory graph and security policies so you don't have to write raw SQL RLS queries for every RAG call.
Where we are now
Rice is currently in closed beta/alpha. We are working with a few design partners in the voice and support automation space who need that sub 100ms retrieval speed.
We know using WNNs for semantic search is a contrarian bet compared to the massive investment in Vector DBs. We are specifically optimizing for "Hot State" (short term, high velocity memory) rather than "Cold Storage" (archival knowledge), though the lines are blurring.
Use Cases we are seeing: - Voice Agents: Shaving 200ms off RAG latency to make conversation feel natural. - Multi-Agent Hand-offs: Agent A (Sales) updates a "Customer Mood" state, and Agent B (Support) sees it instantly without hallucinating. - Internal Tools: Enforcing strict ACLs (e.g., "Junior Devs can't query the Salary Table") at the infrastructure layer.
We are looking for engineers who are pushing the limits of agent latency or struggling with state management to try it out and tell us where it breaks.
I’m especially interested in hearing your skepticism on the WNN approach - we know it’s weird, but for our specific constraints, the speed tradeoff has been worth it.