Adding TTL to an embedded KV store (built on SQLite's B-tree layer, not SQL).
The constraint: TTL needs two conflicting access patterns – lookup by key
(get expiry check) and scan by expiry time (efficient purge). Single index
can't serve both. Using two hidden companion CFs to solve this.
Three open questions:
1. Transparent expiry on get() vs explicit TTL check – which is better UX?
2. Should purge_expired() accept a max_keys limit to bound transaction size?
3. Seconds vs milliseconds precision – is sub-second TTL ever needed?
swaminarayan•1h ago
The constraint: TTL needs two conflicting access patterns – lookup by key (get expiry check) and scan by expiry time (efficient purge). Single index can't serve both. Using two hidden companion CFs to solve this.
Three open questions: 1. Transparent expiry on get() vs explicit TTL check – which is better UX? 2. Should purge_expired() accept a max_keys limit to bound transaction size? 3. Seconds vs milliseconds precision – is sub-second TTL ever needed?
Design doc: https://github.com/hash-anu/snkv/blob/adding_ttl/docs/propos...
For new devs, Current design doc: https://github.com/hash-anu/snkv/blob/adding_ttl/internal/KV...