The blobs are collected over time and accessed in order of insertation but not necessarily from the beginning: multiple clients are 'reading' them starting from different start positions.
According to my research PostgreSQL looks good: - It can store blobs using TOAST - Handle partitioning, removing stalled data and some other neat stuff with the TimescaleDB extension.
The only requirement is to access the blobs as fast as possible. Of course it seems the limiting factor will be disk (and network) latency, but on top of these the DB should add only a fraction of overhead.
Currently i'm storing the blobs in memory and it's very good for latency, but very limiting and prevents adding new features.
I'm very curious whether there's any other option that i missed.
toomuchtodo•1d ago
Benedicht•1d ago