I built a small open-source rate limiter in Go that doesn’t write every hit to storage. It uses a pattern I call the Vector–Scalar Accumulator (VSA): keep the latest committed allowance as a scalar in storage (S) and track recent flux in RAM as a vector (A_net). Availability is computed in O(1) as S - |A_net|. We only persist when the net drift matters (e.g., |A_net| >= threshold), then apply the delta once and reset. In traffic with back-and-forth or retries, this cuts datastore writes by ~95–99% because opposing ops cancel in memory.
Why this is different from batching: batching still ships every event later; VSA collapses commutative updates into their net now. I/O scales with information (signal), not with transactional volume (noise).
What you get
O(1) reads and O(1) memory per key (HashMap<Key, VSA>)
Dramatically fewer writes, locks, and round-trips under high churn
Concurrency-safe TryConsume(n) to prevent oversubscription
Background worker that commits only when a threshold is crossed, plus final flush on shutdown
Pluggable persistence (mock included); keep your DB/Redis calm, or pair with a durable log if you need every event
Trade-offs
Uncommitted in-RAM flux can be lost on crash (tune the threshold to your risk, or mirror raw events to a log)
Best for commutative counters and volatile limits (rate limiting, edge quotas, hot counters, inventory holds)
Why you might care
Running distributed rate limiters, API gateways, or edge caches
Hotspot keys causing write amplification/lock contention
You want durability without paying the price of writing every micro-op
How it works (tiny sketch)
// Always read availability in O(1)
Available = S - |A_net|
// Persist only when signal matters
if |A_net| >= threshold {
S = S - A_net
A_net = 0
}
Repo, docs, and a reference API demo are included. Curious to hear feedback from folks building distributed limiters or edge caches—especially around crash semantics, threshold tuning, and production anecdotes. If you need every event, I recommend pairing VSA with a durable log while keeping the hot path net-based.
etalazz•3mo ago
Why this is different from batching: batching still ships every event later; VSA collapses commutative updates into their net now. I/O scales with information (signal), not with transactional volume (noise).
What you get
O(1) reads and O(1) memory per key (HashMap<Key, VSA>) Dramatically fewer writes, locks, and round-trips under high churn Concurrency-safe TryConsume(n) to prevent oversubscription Background worker that commits only when a threshold is crossed, plus final flush on shutdown Pluggable persistence (mock included); keep your DB/Redis calm, or pair with a durable log if you need every event Trade-offs
Uncommitted in-RAM flux can be lost on crash (tune the threshold to your risk, or mirror raw events to a log) Best for commutative counters and volatile limits (rate limiting, edge quotas, hot counters, inventory holds)
Why you might care
Running distributed rate limiters, API gateways, or edge caches Hotspot keys causing write amplification/lock contention You want durability without paying the price of writing every micro-op How it works (tiny sketch)
// Always read availability in O(1) Available = S - |A_net|
// Persist only when signal matters if |A_net| >= threshold { S = S - A_net A_net = 0 }
Repo, docs, and a reference API demo are included. Curious to hear feedback from folks building distributed limiters or edge caches—especially around crash semantics, threshold tuning, and production anecdotes. If you need every event, I recommend pairing VSA with a durable log while keeping the hot path net-based.