It's also worth noting that the default for data checksums has changed, with some overhead due to that.
I wonder if it's just being executed on a different VMs with slightly different performance characteristics. I can't tell based on the formulation in the post whether all the runs for one test are executed on the same VM or not.
Is it because remote storage in the cloud always introduces some variance & the benchmark just picks that up?
For reference, anarazel had a presentation at pgconf.eu yesterday about AIO. anarazel mentioned that remote cloud storage always introduced variance making the benchmark results hard to interpret. His solution was to introduce synthetic latency on local NVMes for benchmarks.
Has anybody seriously benchmarked this ?
I don’t think io uring would make a difference with this setting but I’m curious, as it’s the default for oracle and sybase.
See e.g. here: https://www.cybertec-postgresql.com/en/postgresql-18-and-bey...
Is anyone familiar with Postgres development able to give an update on the state of the feature? Is it planned for a future (18 or 19) release?
[0]: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit...
[1]: https://www.postgresql.org/docs/devel/app-pgdump.html#:~:tex...
Before Postgres 18 was released, the docs listed `format` as an option for `pg_dumpall` in the upcoming version 18 (e.g. Wayback Machine from Jun 2025 https://web.archive.org/web/20250624230110/https://www.postg... ). The relevant commit is from Apr 2025 (see link #0 in my original comment). But now all mention has been scrubbed, even from the Devel branch docs.
If you care about performance, don't use network storage.
If you are using local nvme disk, then it does not matter if you are using Postgres 17 or 18. Performance is about the same. And significantly faster than network storage.
Am I correct in that using local disk on any VPS has durability concerns?
But most applications run fine from local storage and can tolerate some downtime. They might even benefit from the improved performance. You can also fix the durability and disaster recovery concerns by setting up on RAID/ZFS and maintaining proper backups.
I think only way it could work was if I implemented sync replication like planetscale, but that arduous.
I would never ever trust OVH with any important data or servers, I mean we saw how they secured their datacenters where it took 3h to cut the power while the datacenter was burning.
Also hey this is HN not Twitter I think we can be a bit more civilized. Not a good look imo for a CEO to get that upset over a harmless comment.
But in the cloud, if the physical server backing your instance dies, or even if someone accidentally issues a shutdown command, you don’t get that same drive back when the new instance comes up. So a problem that is normally solved by basic local redundancy suddenly becomes impossible, and thus you must either run synchronous or semi-sync replication (the latter is what PlanetScale Metal does), accepting the latency hit from distributed storage, or asynchronous replication and accept some amount of data loss, which is rarely acceptable.
and that EC2 local NVMe encryption keys are ephemeral is nice against leaks, but not a necessity for other clouds (and not great for resumability, which can really downgrade business continuity scores), and I expect for all the money they ask for it, to be able to keep it relatively secure even across reboots
> IOPS: 3,000
> IOPS: 300,000 for 551$ per month
the cloud is ridiculous.just for reference with 4 consumer nvmes and raid10 and pciex16 you can easily do 3m IOPS for one time cost of like 1000$
in my current job we constantly have to rethink db queries/design because of cloud IOPS, and of course not having control over RDS page cache and numa.
every time I am woken up at night because a seemingly normal query all of the sudden goes beyond our IOPS budget and the WAL starts trashing, I seriously question my choices.
the whole cloud situation is just ridiculous.
lspci is only written in the old alchemy books, in the whispers of the thrice great Hermes.
PS: I have personally put down actual fires in a datacenter, and I prefer it to this 3000 IOPS crap.
Cloud costs are getting large enough that I know I’ve got one foot out the door and a long term plan to move back to having our own servers and spend the money we save on people. I can only see cloud getting even more expensive, not less.
(1) It is shocking how much of a move to the cloud was driven by accountants wanting opex instead of capex, but are now concerned with actual cashflow and are thinking of going back. The cloud is really good at serving web content and storing gobs of data, but once you start wanting to crunch numbers or move that data, it gets expensive fast.
We had a couple rather large datacenters, but both were in the US. The only infrastructure we had in the EU was one small server closet. We had no hosting capacity in Brazil, China, etc. Multi-region availability drove us to the cloud - just not in the "high availability" sense of the term.
When you have three major hyperscalers competing for your dollars this is basically not true and not how markets work...unless they start colluding on prices.
We've already seen reduction in prices of web services costs across the three major providers due to this competitive nature.
Or you just use something like CockroachDB, YugabyteDB etc that auto replicate, auto rebalance if a node goes down, and have build in support for backups to and from S3...
Or if your a bit more hands on, multigress seems to be closing to completion ( https://github.com/multigres/multigres ) from the guy that make Vitess for Mysql.
The idea that managing hardware and software is hard, is silly yet, people (mostly managers it seems ) think its the best solution.
Even when it's done, it's going to be a lot of work to run. sure, it's not guaranteed to be hard, but if it's not your core business and you're making money, having someone else do it gives you time to focus on what matters.
not to mention then you have all the issues people are discussing managing your own backups, snapshots and replication and etc
A quick glance of swapping to the official postgres container shows POSTGRESQL_DATABASE is renamed to POSTGRESQL_DB. The other issue is the volume mount path is currently /bitnami/postgresql.
[1] https://hub.docker.com/_/postgres#pgdata
[2] https://www.postgresql.org/docs/current/upgrading.html#UPGRA...
alberth•3mo ago
6r17•3mo ago
cientifico•3mo ago
I concluded that better IO planning it's only worth it for "slow" I/O in 18.
Pretty sure it will bring a lot of learnings. Postgress devs are pretty awesome.