frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Ask HN: Options for storing blobs for fast access?

2•Benedicht•1d ago
I'm looking for a database to store blobs ranging from a few bytes to a few MBs. Presumably the number of blobs will be in the hundreds of millions. Each blob has well-defined metadata, perfectly fitting in SQL columns.

The blobs are collected over time and accessed in order of insertation but not necessarily from the beginning: multiple clients are 'reading' them starting from different start positions.

According to my research PostgreSQL looks good: - It can store blobs using TOAST - Handle partitioning, removing stalled data and some other neat stuff with the TimescaleDB extension.

The only requirement is to access the blobs as fast as possible. Of course it seems the limiting factor will be disk (and network) latency, but on top of these the DB should add only a fraction of overhead.

Currently i'm storing the blobs in memory and it's very good for latency, but very limiting and prevents adding new features.

I'm very curious whether there's any other option that i missed.

Comments

toomuchtodo•1d ago
https://news.ycombinator.com/item?id=37325379 might be a useful thread for you inquiry.
Benedicht•1d ago
Thanks, i'm reading it right now!

PixWit – All-in-One AI Video Creation Platform

https://pixwit.ai
1•maysunyoung•3m ago•1 comments

Snake Oil by Bruce Schneider (February 15, 1999)

https://www.schneier.com/crypto-gram/archives/1999/0215.html
1•chistev•6m ago•0 comments

A tiny course on differentiable rasterization

https://jjbannister.github.io/tinydiffrast/
1•ibobev•7m ago•0 comments

A Couple 3D AABB Tricks

https://gpfault.net/posts/aabb-tricks.html
2•ibobev•7m ago•0 comments

Hype Without Proof: The Influentists

https://carette.xyz/posts/influentists/
3•weird_trousers•8m ago•1 comments

Bindless Oriented Graphics Programming

https://alextardif.com/BindlessProgramming.html
2•ibobev•8m ago•0 comments

The Fallacy of Cracking Contests by Bruce Schneider (December 15, 1998)

https://www.schneier.com/crypto-gram/archives/1998/1215.html
1•chistev•9m ago•0 comments

Free and local browser tool for designing gear models for 3D printing

https://gears.dmtrkovalenko.dev
1•neogoose•10m ago•0 comments

Turning marketing into software: lessons from building an agent-driven OS

https://www.google.com/search?q=site%3Avect.pro&oq=&gs_lcrp=EgZjaHJvbWUqCQgAECMYJxjqAjIJCAAQIxgnG...
2•MMAFRAZ•15m ago•1 comments

I built a marketing operating system instead of another AI tool

https://www.google.com/search?q=site%3Avect.pro&oq=s&gs_lcrp=EgZjaHJvbWUqCAgDEEUYJxg7MgYIABBFGDwy...
2•afrazullal•17m ago•1 comments

A new type of microscope lets scientists observe life unfolding inside cells

https://www.thebrighterside.news/post/a-new-type-of-microscope-lets-scientists-observe-life-unfol...
1•thunderbong•17m ago•0 comments

PostgreSQL 18 Returning Enhancements: A Game Changer for Modern Applications

https://www.pgedge.com/blog/postgresql-18-returning-enhancements-a-game-changer-for-modern-applic...
1•todsacerdoti•18m ago•0 comments

Netflix Ruined Korean Dramas Forever [video]

https://www.youtube.com/watch?v=p1_j6izmEX4
1•mgh2•18m ago•0 comments

Show HN: Claude Code Skill for Deep Code Reviews

https://github.com/turingmindai/turingmind-code-review
1•vinkupa•19m ago•0 comments

I Made Visualizing LLM Model Collapse at Gen 20

https://github.com/mhh1430hacker/Ainex-Limit-Experiment
1•Mhh1430•20m ago•0 comments

Ask HN: Help me crowd source this directory of text-only news sites

1•keepamovin•21m ago•0 comments

Climbing the mountain: or, venturing into PL theory

https://techne98.com/blog/climbing-the-mountain/
2•fixedprog•25m ago•0 comments

Empire Strikes Back (At Privacy): Archaeology of Tracking on Government Websites

https://www.flux.utah.edu/paper/singh-pets26
1•pabs3•26m ago•0 comments

Nineteen Septillion Addresses – Setting Up an ASN and Getting an IPv6 Allocation

https://alastairbarber.com/Setting-Up-ASN-IPv6-Routing-BIRD-Teltonika-Router-Wireguard/
2•alibarber•32m ago•0 comments

The Q, K, V Matrices

https://arpitbhayani.me/blogs/qkv-matrices/
4•yashsngh•35m ago•0 comments

My Logitech mouse became unusable, Logi Options+ can't validate certificate

2•enescakir•35m ago•3 comments

Quaject

https://en.wikipedia.org/wiki/Quaject
2•gjvc•38m ago•0 comments

Modern C++ Programming (open-access course)

https://federico-busato.github.io/Modern-CPP-Programming/
2•pjmlp•41m ago•0 comments

OnceMap: Rust Pattern for Running Concurrent Work Exactly Once (uv codebase)

https://codepointer.substack.com/p/uv-oncemap-rust-pattern-for-running
1•hkmaxpro•41m ago•0 comments

Why one small American town won't stop stoning its residents to death

https://archiveofourown.org/works/73396436?view_adult=true
1•barry-cotter•41m ago•0 comments

Photonic Raises $130M to Accelerate Quantum Computing and Networking

https://photonic.com/news/photonic-raises-180m/
1•jc_811•42m ago•0 comments

Necrosecurity, Immunosupremacy, and Survivorship in the Imagination of Covid-19

https://www.degruyterbrill.com/document/doi/10.1515/opan-2020-0104/html
1•colinprince•43m ago•0 comments

A Glimpse into DexProtector

https://www.romainthomas.fr/post/26-01-dexprotector/
1•shelfchair•44m ago•0 comments

This Month in Redox – December 2025

https://www.redox-os.org/news/this-month-251231/
4•Ganipote•55m ago•0 comments

Show HN: Qpost.dev – An open API to post to TikTok, YT, and IG in one place

https://qpost.dev
1•arslan2012•55m ago•0 comments