frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Building an HTML-first site doubled our users overnight

https://mohkohn.co.uk/writing/html-first/
412•edent•3h ago•172 comments

AMA: I'm Eric Ries (The Lean Startup) & Author of New Bestseller Incorruptible

79•eries•1h ago•54 comments

Apache Burr: Build reliable AI agents and applications

https://burr.apache.org/
24•anhldbk•47m ago•4 comments

Mercedes‑Benz starts large‑scale production of electric axial flux motor

https://media.mercedes-benz.com/en/article/bebac2af-acdc-465a-9538-adb0bf3d8ccf
342•raffael_de•8h ago•196 comments

PgDog is funded and coming to a database near you

https://pgdog.dev/blog/our-funding-announcement
95•levkk•1h ago•58 comments

All 9,300 Japanese train station, animated by the year it opened (1872–2026)

https://jivx.com/eki
91•momentmaker•3h ago•33 comments

macOS Container Machines

https://github.com/apple/container/blob/main/docs/container-machine.md
1028•timsneath•15h ago•358 comments

Claude Fable 5

https://www.anthropic.com/news/claude-fable-5-mythos-5
2460•Philpax•22h ago•1954 comments

Buy a train, bridge or tracks from the Swiss Railway

https://sbbresale.ch/
94•kisamoto•2d ago•51 comments

Who Runs Your Rust Future? Hands-On Intro to Async Rust

https://aibodh.com/posts/async-rust-chapter-1-hands-on-intro-to-async-rust/
52•febin•2d ago•5 comments

A €0.01 bank transfer could compromise a banking AI agent

https://blue41.com/blog/how-we-helped-bunq-secure-their-financial-ai-assistant/
47•tvissers•2h ago•21 comments

Smudging the game disc to make speedrunning 'SpongeBob' faster

https://www.inverse.com/input/gaming/the-dirty-secret-that-makes-speedrunning-on-spongebob-a-lot-...
9•pncnmnp•13h ago•2 comments

'They take you out of life, out of time': a journey into Spain's cave paintings

https://www.theguardian.com/science/2026/jun/02/journey-into-spain-palaeolithic-cave-paintings-al...
19•NaOH•1d ago•3 comments

Reviving Papers with Code

https://paperswithcode.co/
131•nielz_r•2d ago•27 comments

AWS Bedrock to require sharing data with Anthropic for Mythos and future models

293•TomAnthony•7h ago•184 comments

The iPad was on Tailscale: a WebRTC debugging story

https://p2claw.com/blog/2026-06-09-the-ipad-was-on-tailscale/
6•syllogistic•35m ago•0 comments

Hacking for Defense Stanford 2026 – Lessons Learned Presentations

https://steveblank.com/2026/06/08/g-for-defense-stanford-2026-lessons-learned-presentations/
59•sblank•1d ago•30 comments

I Hate (Most) Keyboard 'Fn' Keys

https://danq.me/2026/06/09/fn-keys/
130•speckx•2h ago•136 comments

Upcoming breaking changes for npm v12

https://github.blog/changelog/2026-06-09-upcoming-breaking-changes-for-npm-v12/
443•plasma•18h ago•180 comments

Magnetoelectric antennas could transform how underwater robots talk

https://newatlas.com/engineering/magnetoelectric-antennas-submarine-robots-communications/
55•breve•3d ago•22 comments

German ruling declares Google liable for false answers in AI Overviews

https://the-decoder.com/landmark-german-ruling-declares-googles-ai-overviews-are-googles-own-word...
830•ahlCVA•14h ago•464 comments

Rich Sutton on AI creativity and discovery

https://twitter.com/RichardSSutton/status/2061216087744946656
184•yimby•13h ago•94 comments

RIP software hackathons. Long live the hardware hackathon

https://blog.oscars.dev/posts/rip-software-hackathons-long-live-the-hardware-hackathon/
243•ozcap•17h ago•117 comments

Surprise, pay $1000

https://forestwalk.ai/blog/surprise-blacksmith-costs/
300•apike•17h ago•141 comments

US Consumer Price Index up 4.2%

https://www.bls.gov/news.release/cpi.nr0.htm
19•ortusdux•37m ago•3 comments

Notes on DeepSeek

https://twitter.com/NikoMcCarty/status/2064686557400100884
55•vinhnx•1h ago•37 comments

Show HN: macOS menu bar gauges for your Claude Code quota

https://github.com/grzegorz-raczek-unit8/claude-quota
48•grzracz•6h ago•31 comments

What it feels like to work with Mythos

https://www.oneusefulthing.org/p/what-it-feels-like-to-work-with-mythos
341•swolpers•22h ago•302 comments

Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks

https://aarushgupta.io/posts/kan-fpga/
267•ag2718•20h ago•45 comments

I thought I knew how electrolysis worked [video]

https://www.youtube.com/watch?v=eq7fR9ISuCw
88•tambourine_man•5d ago•10 comments
Open in hackernews

PgDog is funded and coming to a database near you

https://pgdog.dev/blog/our-funding-announcement
95•levkk•1h ago

Comments

htrp•1h ago
>PgDog is a sharder, connection pooler and load balancer for PostgreSQL. Written in Rust, PgDog is fast, reliable and scales databases horizontally without requiring changes to application code.

Still trying to figure out how this works technically, is the performance gain really just re-write in rust?

levkk•1h ago
Not quite. The performance gain is to bring those features to Postgres!

Edit:

Performance gains are from having the ability to load balance reads (horizontal scaling for read queries) and scale out writes (with sharding). Once instance bottleneck in Postgres has many faces:

1. Behind schedule vacuums because of too many dead tuples (too many writes)

2. The WALWriter is single-threaded and IO-bound - Postgres can only do about 200-300MB/sec in writes per instance (real prod numbers on EC2 with NVMes and ZFS, basically best case scenario).

3. Bulkheading: single primary is a single point of failure. With 12 primaries, if one fails, 91% of your customers don't notice.

The list goes on. Rust is just a side effect. We love it because it's fast and correct - the perfect match for a database product.

VeninVidiaVicii•1h ago
Oh thanks for clearing that up.
levkk•1h ago
Sorry, out walking the dog (not a pun). I'll post more details in a few.
Pet_Ant•1h ago
I hope people pronounce this as „pig-dog” and has a mascot that looks like „man-bear-pig”
levkk•1h ago
Crap! Missed opportunity.
fulafel•1h ago
Does making it "just work" here come with any caveats vs standard PG?
levkk•1h ago
Getting there! Cross-shard writes do because of 2pc. Reads are eventually consistent.
danielheath•1h ago
Given that they implement connection pooling and sharding, I'm going to say "not at all".

You _could_ make that ACID, but it's not going to be faster than a single machine.

melon_tsui•1h ago
2M qps in production is legit. Curious how much RAM and CPU that takes on average per deployment though
levkk•56m ago
Depends. Only pooling, very little. Load balancing/sharding needs to parse queries, so a bit more. Could go up to a GB per pod, sometimes more if you have a lot of unique SQL queries (unique by text, not by parameters). We cache query ASTs to avoid parsing them on each request - that's the bulk of memory usage.
parthdesai•16m ago
Semi related question - I have always wondered, how do you tackle OOM issues at the proxy layer, i.e. let's say a particular SQL query requires proxy to fan out the query to multiple shards, which return a pretty large dataset. I'm assuming you would need to load this dataset in the ram to perform certain operations. What happens if the resulting dataset causes the proxy pod to go OOM?
levkk•13m ago
Two schools of thought:

1. Let it crash. Increase the RAM, try again.

2. Page to disk (swap), make it slow but ultimately work.

Both have their trade-offs. There is no free lunch here.

skiwithuge•1h ago
we are using PG bouncer in production. Interesting, I will follow the evolution of this project
kjuulh•1h ago
I tried out PgDog a while ago, but couldn't find a good way of handling the config except for having this users / pgdog toml file, which makes it a bit awkward to handle in kubernetes where we often do multi-tenancy in postgres - or rather having many databases on the same instance(s), and have them come and go at will.

Also had an issue with it because it cached authentication requests when doing passthrough it seems, I'd changed the roles password, but it kept using the old one, which was no bueno ;).

PgDog seems to make more sense when you really care about a few databases that need massive scale, rather than a simple proxy in front of postgres. I'll keep following the development though, it is much needed in this space, postgres can use all the investment it can get to get it past the single machine scale that it excels at currently.

maherbeg•58m ago
Happy to chat about this, but we use the AWS secrets manager flowing into External Secrets Operator to generate a pgdog_users.toml. We then kick off a workflow to refresh things, but our rate of change here is much smaller than a super dynamic multi-tenant system.

You could also build a watcher side car that watches for changes of the pgdog_users.toml and have pgdog refresh itself then too with this combination. We thought about that but prefer to control the reloads for our needs.

jeremyjh•1h ago
It’s surprising they don’t mention advantages over other sharding systems like Citus. Maybe it’s just the fact that it’s only a proxy and not core extensions? But that could limit capabilities.
levkk•55m ago
We do, just buried deep in our blog: https://pgdog.dev/blog/pgdog-vs-citus

The same old processes vs. threads debate, plus having the ability to scale the coordinator past a single machine. So, if you're OLTP, definitely consider PgDog. OLAP - Citus still wins because of its advanced query engine. We'll get there.

ParadisoShlee•1h ago
I've moved from pgbouncer to pgdog a few months ago without issue. Huge fan.
moralestapia•1h ago
Cool work, thanks.

Wrt. the pooler, how do you compare with pgbouncer?

I'm interested because I have a postgres instance, low-traffic but still like ... tens of r(eads)ps. I was not running anything close to the machine limits but still added pgbouncer to improve performance and didn't see a noticeable difference. I was stress-testing the machine obv., I'm not talking about the 10 rps, lol.

For context, my numbers were something like 10k rps +/- 1k vanilla postgres and like 9k rps +/- 1k with pgbouncer in front of it. So ... slightly slower but big error bars so I wouldn't say for sure. I ended up not using pgbouncer as the benefit was immaterial.

Also yeah, in case you want to check it out, it's the db that backs this project: https://httpstate.com.

levkk•57m ago
Old benchmark, but still good: https://pgdog.dev/blog/pgbouncer-vs-pgdog
faangguyindia•1h ago
i am not using any tool like pgbouncer and have not run into any issues so far. Is it even required these days? Have you guys tested your setup without these connection poolers/multiplexers?
rswail•23m ago
Each connection is a process on the server, that takes up both CPU and RAM, it will run out.

This solves the thousands of clients case for read in a way that is transparent to the clients.

Yes it's required at large scale, especially if you want to distribute reads or shard to a particular geographical area.

simonw•1h ago
Suggestion: have more than just helm and Docker in your quickstart documentation. I'd like to try this out just to see what it can do, but not quite enough to fire up one of those systems for it.

Is there a binary I can run directly?

levkk•58m ago
We should add it to brew/apt/etc for sure. Also, we could add it to crates.io so you could do something like `cargo install pgdog`. Distribution, distribution, distribution.
simonw•45m ago
I also appreciate GitHub releases with pre-compiled binaries for different platforms. The more options the better!
e12e•47m ago
In addition - the docker compose example doesn't set up any data volumes for the postgres instances - that might be considered a bug?

Then again, sharding on a single host probably isn't very useful anyway - but it might work with docker in swarm mode?

levkk•14m ago
The docker compose example is just a demo. I don't know anyone who runs Postgres with docker compose / swarm in prod :) But yes, happy to add volumes so it seems more real.
bourbonproof•1h ago
the reason mongo is a joy to use in scaled env is because no additional setup/software needed and all drivers natively support secondary/primary writes/reads and topological changes. so it's end to end, and adding is as a new proxy in frontend of postgres leads to all clients being incompatible or the code itself has no control anymore about when to use a secondary and what allowed stall is acceptable for a particular query. Any solutions to this by pgdog?
saghm•55m ago
> all drivers natively support secondary/primary writes/reads and topological changes.

Expanding on that a bit, mongo drivers even have a shared specification of the state machine for monitoring topology changes[1] and algorithm for selecting the server to send an operation to[2] (along with various declarative test cases that the drivers use to validate them alongside the specs in the repo). I think people sometimes underestimate how important the client-side work is to this sort of experience; for all of the faults mongo has had over the years, the amount of investment that they put into the client libraries is something I've never seen anywhere else (although having spent several years working on some of these libraries, my take is likely very biased).

[1]: https://github.com/mongodb/specifications/blob/master/source... [2]: https://github.com/mongodb/specifications/blob/master/source...

maherbeg•58m ago
I'm a big PGDog fan! It really helped us scale our connection proxy needs pretty substantially and it has great features like auto mode to support Aurora failovers neatly. It's infra that just works.
chrisvenum•54m ago
I am trying to gain a basic understanding of this: Right now I have a 4TB DB on one large box. Is the idea that using a proxy tool like PGDog I could spin up 8 smaller boxes handling ~500GB each and then one medium box for the proxy?

Right now I have a project that has very heavy write traffic from multiple services and a web app that reads from this. We are starting to hit the point where no amount of indexing, query optimisation, caching or box upgrades is helping us. We are looking at maybe moving the bulk of the static data to clickhouse to reduce the DB size but I would love to hear if PgDog or other kind of sharding could be useful for this use case.

999900000999•53m ago
How are 3 developers going to QA this properly ?
pantulis•38m ago
How are 3 developers going to sell that to any company? Procurement will have a field day.
rswail•16m ago
They have funding. That's what it will be for. I wish them well and appreciate that people are still doing FOSS.

As long as they don't get undercut by the equivalent of AWS https://aws.amazon.com/rds/proxy/ which is a managed pgbouncer.

999900000999•6m ago
The issue is if the DB layer fails your product is going to completely stop working.

You’d need a ton of faith in these 3 people.

Feels more like it would work better inside of a bigger organization.

The QA tester in me is kinda risk adverse.

yabones•52m ago
I'm curious how this might help with our biggest downtime-causer with postgres, which is major version upgrades. Poolers do a great job for failover and load balancing, but we consistently need ~10-20 minutes of downtime once or twice a year to do upgrades. Logical replication between old->new versions could probably help, but it would still require flipping everything over to the new cluster without partial writes or anything silly. Anybody have experience with this?
boxed•45m ago
Seconded. Coming from MySQL this is a huge regression that makes Postgres look like something from the 80s. I still wonder why this isn't seen as the absolutely highest priority.
Blackthorn•38m ago
Probably because it's an open source project and apparently none of its users cared about this feature enough to develop it or fund it.
jeltz•37m ago
What would the feature be? There are things which can be improved in this area but there is no way to do major version upgrades for free. The reason it is like this in PostgreSQL, and MySQL, is due to tradeoffs. Making major version upgrade more complex allows for making other things simpler.
jeltz•38m ago
I have not ran MySQL for some years but it at least used to have exactly the same issue. Upgrading a database with MySQL can take a long time if you have many tables. The main difference is only really that PostgreSQL does it with a separate tool, pg_upgrade, while MySQL does it as part of the main binary.

For both MySQL and PostgreSQL you will need to use some kind of logical upgrades if you want no downtime.

Ozzie_osman•48m ago

  We sharded over 20 TB that we know about.
This is probably a typo, right? 20TB isn't that big. I would imagine they've sharded a lot more than that
GiorgioG•34m ago
For a vast majority of use cases 20TB is positively enormous.
tingletech•29m ago
that article seems to suggest 20TB total over the dozen deployments in prod.
happyopossum•26m ago
Sure, but 20TB in “the only database you need” is mere hours or minutes worth of data for many workflows.
returningfory2•23m ago
This product is for Postgres deployments that are so large they need to be sharded. For these use cases, I think 20TB is about normal.
jeltz•2m ago
Yes. But for most workloads it is not much for PostgreSQL. You often will not have to shard at all.
mplanchard•
drchaim•44m ago
Good stuff, although I’m not quite sure about the fast OLAP use case.

If you’re already sharding by tenant for other reasons, OK… But I see CDC to a true OLAP system as more scalable.

PostgreSQL still needs real columnar tables in the core, hopefully one day

levkk•2m ago
OLAP means different things to different people. For us, it's just making sure your admin dashboard keeps working basically:

  SELECT tenant_id, COUNT(clicks)
  FROM users
  GROUP BY tenant_id
  ORDER BY 2 DESC
  LIMIT 25;
Performance is a side effect - definitely needed and we'll do everything we can, but we are not competing with ClickHouse or Snowflake - just trying to make sharded Postgres work with your app.
mnbbrown•39m ago
I've loved using pgdog for the last 6 months. It's been incredibly stable. It's nifty how they've solved the LISTEN/NOTIFY on a transaction pooler problem.
tschellenbach•17m ago
PgDog, Neki, multigres, awesome to see. And yes this is the main issue with postgres. Well this and not having index hints, looking forward to 19
tomnipotent•19m ago
MySQL has advocated for decades spinning up a replica with the upgraded version, waiting for it to catch up to master before promoting it to the new master. You can do the same thing with Postgres.
jeltz•4m ago
Exactly, MySQL and PostgreSQL are the same here. Maybe one is a bit faster than the other at doing major version upgrades but the behaviours are quite similar.
tempest_•35m ago
We use logical replication and a pause / swap in pgbouncer for ~5s of paused (but not failed) writes.

This is for DBs that are ~1-1.5TB but doesnt have a huge amount of churn/qps

Effectively what is described here https://www.pgedge.com/blog/always-online-or-bust-zero-downt...

tux3•23m ago
Logical replication is how this is typically done. If you have some infra-as-code setup, you create a new cluster with identical settings except for the major version, import the schema, start copying data from a read-replica running the old version, stop accepting writes from the old version (downtime starts), sync the sequence numbers, and point your services to the new cluster (downtime ends).

If you use something like CloudNativePG they automate parts of the process with cli tools and declarative syntax. Otherwise you take the time to figure it out by hand. It might sound complicated, but just practice on your staging DB, and if all goes well you do the same procedure in prod.

Edit: Apparently Postgres 19 has a patch for one-shot logical replication of sequences! https://www.depesz.com/2025/11/11/waiting-for-postgresql-19-...

tschellenbach•16m ago
Logical replication solves this. You roll the cluster, downtime is minimal. like 60s maybe.
1m ago
RDS caps out at 64 TB unless you use Aurora, so 20 TB is totally manageable without sharding.
rbranson•12m ago
You are correct. As a point of comparison: almost ten years ago at Segment we had a single Aurora PostgreSQL instance with ~50T of data, it was used to index potential identity data in a much larger corpus of files stored in S3.