frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Just make it scale: An Aurora DSQL story

https://www.allthingsdistributed.com/2025/05/just-make-it-scale-an-aurora-dsql-story.html
134•cebert•1y ago

Comments

glzone1•1y ago
Early dsql had some weird limits I think - anyone actually using in production with feedback on current corners and limits?
Marbling4581•1y ago
I don't use it, but have been keeping an eye on it.

At launch, they limited the number of affected tuples to 10000, including tuples in secondary indexes. They recently changed this limit to:

> A transaction cannot modify more than 3,000 rows. The number of secondary indexes does not influence this number. This limit applies to all DML statements (INSERT, UPDATE, DELETE).

There are a lot of other (IMO prohibitive) restrictions listed in their docs.

https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...

pzduniak•1y ago
Who would use Preview products in production? I'm building out some software that would fit perfectly into the constraints set for DSQL, but I realistically can't commit to something with no pricing / guarantees.
EwanToo•1y ago
This blog post appears to be part of the scheduled launch marketing, it's now generally available

https://aws.amazon.com/blogs/aws/amazon-aurora-dsql-is-now-g...

mjb•1y ago
Which features would you like to see the team build first? Which limits would you like to see lifted first?

Most of the limitations you can see in the documentation are things we haven't gotten to building yet, and it's super helpful to know what folks need so we can prioritize the backlog.

avereveard•1y ago
indexes! vector, trigram and maybe geospatial. (some may be in by now I didn't follow the service as closely as others)

note, doesn't have to be pg_vector pg_trgm or PostGIS, just the index component even if it's a clean room implementation would make this way more useful.

loginatnine•1y ago
Views and foreign keys!
mjb•1y ago
Thanks. The team's working on both. For views, do you need updatable views, or are read-only views sufficient?
loginatnine•1y ago
For me it's RO views.
tigy32•1y ago
I believe views were added to the preview a little while ago

edit from the launch: "With today’s launch, we’ve added support for AWS Backup, AWS PrivateLink, AWS CloudFormation, AWS CloudTrail, AWS KMS customer managed keys, and PostgreSQL views."

loevborg•1y ago
Which ones? It seems eminently usable from the outside now, at least for greenfield work. The subset of Postgres it supports is most of good/core/essential Postgres. (But I haven't tried it)
geodel•1y ago
Good read. I like the part that both writing low level as well as high level component in Rust was proven worthwhile.

Maybe one can transform slow code from high level languages to low level language via LLMs in future. That can be nice performance boost for those who don't have Amazon engineers and budgets

SahAssar•1y ago
> Maybe one can transform slow code from high level languages to low level language

I think you are describing a compiler?

geodel•1y ago
I mean reading this article:

1) Kotlin code --> Java byte code --> JVM execution (slow)

vs

2) Kotin code --> Rust/Zig code --> Zig compiler --> native execution (fast)

Compiler is involved in both cases but I was thinking of 2) where slower code in high level lang is converted to another lang code. The compiler of which is known to produce fast runinng code.

dhosek•1y ago
You’re describing a transpiler, but the problem is that idioms in a GC language like Kotlin don’t necessarily translate to a non-GC language like Rust or Zig. Add in the fact that Rust doesn’t have OO inheritance which is essential for a lot of JVM code to work (I don’t know much about Zig) and I’d be very suspicious of code generated by a Kotlin to Rust transpiler. (On the other hand, one of the first transpilers I ever encountered, web2c, worked well because the source language, Pascal, could be fairly easily translated into functional C without much if any sacrifice of speed or accuracy.)
mjb•1y ago
Demiurge•1y ago
Many interesting things, for instance, I've been hearing a lot about how fast Java is, that it can be as fast as C++, and then I see this:

> But after a few weeks, it compiled and the results surprised us. The code was 10x faster than our carefully tuned Kotlin implementation – despite no attempt to make it faster. To put this in perspective, we had spent years incrementally improving the Kotlin version from 2,000 to 3,000 transactions per second (TPS). The Rust version, written by Java developers who were new to the language, clocked 30,000 TPS.

I feel like there is more to this, like some kind of a bottleneck, memory footprint, some IO overhead?

> Our conclusion was to rewrite our data plane entirely in Rust.

The point is well taken, figuring it out is not worth it, if you can just "rewrote" or have green field projects.

> These extension points are part of Postgres’ public API, allowing you to modify behavior without changing core code

Also, interesting. So PostgreSQL evolved to the point that it has a stable API for extensibility? This great for the project, maintain a modular design, and some stable APIs and, you can let people mix and match and reduce duplication of effort.

anarazel•1y ago
> So PostgreSQL evolved to the point that it has a stable API for extensibility?

Not across major versions, no. I seriously doubt we will ever make promises around that. It would hamper development way too much.

Demiurge•1y ago
I see, then they're probably saying they found the internal APIs that are just more naturally stable, perhaps because they are close to the APIs used for extensions.
ramanh•1y ago
> I feel like there is more to this, like some kind of a bottleneck, memory footprint, some IO overhead?

blocking/nonblocking IO can explain this numbers

karl_p•1y ago
The JVM can relocate memory to avoid fragmentation. Rust can't, at least natively. Are they not worried about this regression?
geodel•1y ago
Well Java need it because it fragments memory a lot. With Rust one has value types and stack allocation which takes care of one of the biggest cause of fragmentation.
kikimora•1y ago
Writing code that would not fragment memory over time is arguable much harder than writing GC friendly code.
geodel•1y ago
Yeah, cooking food in kitchen is much harder than having it delivered from restaurant at doorstep.

Reasonable people will see if cost makes it worthwhile.

tigy32•1y ago
I haven't found that to be the case in my experience: just for example in java you tend to end up with essentially a lot of `Vec<Box<Thing>>` which causes a lot of fragmentation. In rust you tend to end up with `Vec<Thing>` where `Thing`s are inlined. (And replace Vec with the stack for the common case). I find it more like Java is better at solving a problem it created by making everything an object.
mrkeen•1y ago
With 10x the throughput (TPS) and the lack of GC pauses (which were the cause of the rewrite), how would they measure such a regression, let alone worry about it?
kondro•1y ago
It would be really great to get more context on what a DPU is for pricing: https://aws.amazon.com/rds/aurora/pricing/

I understand that AWS did one TPC-C 95/5 read/write benchmark and got 700k transactions for 100k DPUs, but that’s not nearly enough context.

There either needs to be a selection of other benchmark-based pricing (especially for a primarily 50/50 read/write load), actual information on how a DPU is calculated or a way to return DPU per query executed, not just an aggregate CloudWatch figure.

We were promised DSQL pricing similar to DynamoDB and insofar as it’s truly serverless (i.e. no committed pricing) they’ve succeeded, but one of the best parts of DynamoDB is absolute certainty on cost, even if that can sometimes be high.

belter•1y ago
> one of the best parts of DynamoDB is absolute certainty on cost

That depends if its On Demand or Provisioned, even if they recently added On Demand limits.

kondro•1y ago
You still have absolute certainty. Read or write x amount of data and it will use exactly y R/WCU.

It then just becomes a modeling problem allowing you to determine your costs upfront during design. That’s one of the most powerful features of the truly serverless products in AWS in my opinion.

ejkra•1y ago
Absolute certainty is challenging with a cost-based optimizer in the mix. DDB doesn't face this challenge. Although, cost for some query patterns in DDB would shift into your application layer - so you may not have exactly the cost certainty you imagine?

Would you be willing to pay more for certainty? E.g. rent the full server at peak + 20% and run at 15% utilization some of the time? Provisioned capacity or pre-committed spend seem like reasonable, but perhaps more costly, ways to get certainty.

mrkeen•1y ago
Where can I go to read about distributed SQL and big JOINs or WHERE IN clauses? I was hoping this article would cover that elephant in the room, rather than Rust being significantly more performant than JVM languages.
louis-paul•1y ago
Marc Brooker has written and spoken about DSQL quite a bit. It’s still rather high level. I’d expect one or more papers to come out in the next few months, similarly to other Amazon databases.

https://brooker.co.za/blog/2025/04/17/decomposing.html (includes talk)

https://brooker.co.za/blog/2024/12/03/aurora-dsql.html

https://brooker.co.za/blog/2024/12/04/inside-dsql.html

https://brooker.co.za/blog/2024/12/05/inside-dsql-writes.htm...

https://brooker.co.za/blog/2024/12/06/inside-dsql-cap.html

https://brooker.co.za/blog/2024/12/17/occ-and-isolation.html

mrkeen•1y ago
That's a lot of links for 0 info on distributed JOIN or WHERE IN.
mjb•1y ago
Correct: https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...
tomComb•1y ago
The lack of JSONB is what stopped me.
sgarland•1y ago
Why does it not support TRUNCATE?
jashmatthews•1y ago
My understanding is the way Aurora DSQL distributes data widely makes bulk writes extremely slow/expensive. So no COPY, INSERT with >3k rows, TRUNCATE etc
sgarland•1y ago
TRUNCATE is DROP TABLE + CREATE TABLE, it’s not a bulk delete. It bypasses the typical path for writes entirely.
> Maybe one can transform slow code from high level languages to low level language via LLMs in future.

This is one of the areas I'm most excited for LLM developer tooling. Choosing a language, database, or framework is a really expensive up-front decision for a lot of teams, made when they have the least information about what they're building, and very expensive to take back.

If LLM-powered tools could take 10-100x off the cost of these migrations, it would significantly reduce the risk of early decisions, and make it a ton easier to make software more reliable and cheaper to run.

It's very believable to me that, even with today's model capabilities, that 10-100x is achievable.

geodel•1y ago
I remember many years back one of Go language author wrote C to Go trasformer and used that to convert all compiler, runtime, GC etc into Go.

Now in today's time some experts like above could create base transformer for high level language and frameworks to low level language and frameworks and this all get exposed via llm interfaces.

One can say why all this instead of generating fast binary directly from high level code. But generating textual transformation would give developers opportunity to understand, tweak and adjust transformed code which generating direct binary would not.

bee_rider•1y ago
Python -> C -> Assembly

Probably looks a lot like

Pseudocode -> C -> Assembly

Although the first is easier to run tests on and compare the outputs.

CRISPR tech selectively shreds cancer cells, including "undruggable" cancers

https://innovativegenomics.org/news/crispr-technique-selectively-shreds-cancer-cells/
586•gmays•7h ago•154 comments

Swift at Apple: Migrating the TrueType hinting interpreter

https://www.swift.org/blog/migrating-truetype-hinting-to-swift/
67•DASD•2h ago•26 comments

How to setup a local coding agent on macOS

https://ikyle.me/blog/2026/how-to-setup-a-local-coding-agent-on-macos
175•kkm•4h ago•57 comments

Mmorpg World of ClaudeCraft, vibe coded with Fable 5

https://worldofclaudecraft.com/
43•beatthatflight•1h ago•32 comments

Malware developers added nuclear and biological weapons text to to their spyware

https://twitter.com/jsrailton/status/2064661778978533571
230•marc__1•1d ago•156 comments

Pirates, a naval warfare game inspired by Sid Meier's Pirates

https://piwodlaiwo.github.io/pirates/
149•iweczek•5h ago•65 comments

Palantir loses legal challenge against Swiss investigative magazine

https://www.ft.com/content/7ffcace7-9dc0-4e7e-9912-895ac073f979
86•sschueller•1h ago•13 comments

Slightly reducing the sloppiness of AI generated front end

https://envs.net/~volpe/blog/posts/reduce-slop.html
150•FergusArgyll•7h ago•102 comments

Launch HN: BitBoard (YC P25) – Analytics Workspace for Agents

https://bitboard.work/
29•arcb•5h ago•17 comments

Can I Buy Your KV Cache?

https://arxiv.org/abs/2606.13361
18•MediaSquirrel•2h ago•11 comments

Introduction to UEFI HTTP(s) Boot with QEMU/OVMF

https://blog.yadutaf.fr/2026/06/12/introduction-to-uefi-https-boot-qemu-ovmf/
66•jtlebigot•7h ago•21 comments

"Don't You Just Upload It to ChatGPT?"

https://correresmidestino.com/dont-you-just-upload-it-to-chatgpt/
220•speckx•4h ago•196 comments

Where Did Earth Get Its Oceans? Maybe It Made Them Itself

https://www.quantamagazine.org/where-did-earth-get-its-oceans-maybe-it-made-them-itself-20260612/
92•ibobev•6h ago•55 comments

A PDF that changes based on how its read

https://sgaud.com/texts/pdf
106•SarthakGaud•5h ago•56 comments

I Am Not a Reverse Centaur

https://blog.miguelgrinberg.com/post/i-am-not-a-reverse-centaur
227•ibobev•4h ago•160 comments

Cosmodial Sky Atlas

https://killedbyapixel.github.io/Cosmodial/
22•memalign•3h ago•3 comments

There Is Life Before Main in Rust

https://grack.com/blog/2026/06/11/life-before-main/
62•mmastrac•1d ago•16 comments

Maxproof

https://arxiv.org/abs/2606.13473
123•ilreb•10h ago•10 comments

EV demand up 50% in France and Germany since Iran war

https://www.reuters.com/business/renault-electric-vehicle-orders-have-surged-since-start-iran-war...
67•a_paddy•3h ago•20 comments

If you are asking for human attention, demonstrate human effort

https://tombedor.dev/human-attention-and-human-effort/
1458•jjfoooo4•23h ago•455 comments

Most Beautiful Will Ever Made (1936)

https://paperspast.natlib.govt.nz/newspapers/DOM19360307.2.43
18•cf100clunk•4h ago•9 comments

A dumpster arrived behind my university's library

https://yalereview.org/article/sheila-liming-the-end-of-books
143•mooreds•7h ago•129 comments

Hazel (YC W24) Is Hiring a Full Stack Engineer

https://www.ycombinator.com/companies/hazel-2/jobs/3epPWgu-full-stack-engineer-ts-sci
1•augustschen•9h ago

Show HN: StackScope – I crawled over 40k indie launches to see what they ship

https://stackscope.dev/
38•datafreak_•6h ago•12 comments

WASI 0.3

https://bytecodealliance.org/articles/WASI-0.3
216•mavdol04•8h ago•85 comments

A Call to Action: Stop the FCC's KYC Regime

https://blog.lopp.net/call-to-action-stop-the-fcc-kyc-regime/
300•FergusArgyll•7h ago•198 comments

Encrypted Spaces An architecture for collaborative applications

https://encryptedspaces.org/
60•_____k•9h ago•9 comments

Looking Forward to Postgres 19: It's About Time

https://www.pgedge.com/blog/looking-forward-to-postgres-19-its-about-time
126•xngbuilds•5h ago•37 comments

Nobody ever gets credit for fixing problems that never happened (2001) [pdf]

https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.pdf
715•sam_bristow•21h ago•241 comments

Kimi K2.7-Code: open-source coding model with better token efficiency

https://huggingface.co/moonshotai/Kimi-K2.7-Code
392•nekofneko•11h ago•208 comments