Replacing Protobuf with Rust to go 5 times faster

https://pgdog.dev/blog/replace-protobuf-with-rust

44•whiteros_e•2h ago

Comments

nottorp•2h ago

Are they sure it's because Rust? Perhaps if they rewrite Protobuf in Rust it will be as slow as the current implementation.

They changed the persistence system completely. Looks like from a generic solution to something specific to what they're carrying across the wire.

They could have done it in Lua and it would have been 3x faster.

consp•1h ago

If they made the headline something on the line of "replacing protobuf with a native, optimized implementation" would not get the same attention as putting rust in the title to attract the everything-in-rust-is-better crowd.

desiderantes•1h ago

That never happens. Instead, it always attracts the opposite group, the Rust complainers, where they go and complain about how "the everything-in-rust-is-better crowd created yet another fake headline to pretend that Rust is the panacea". Which results in a lot of engagement. Old ragebait trick.

misja111•1h ago

Correct, this has very little to do with Rust. But it wouldn't have made the front page without it.

locknitpicker•1h ago

Yes you are absolutely right. The article even outright admits that Rust had nothing to do with it. From the article:

> Protobuf is fast, but not using Protobuf is faster.

The blog post reads like an unserious attempt to repeat a Rust meme.

alias_neo•1h ago

I was equally confused by the headline.

I wonder if it's just poorly worded and they meant to say something like "Replacing Protobuf with some native calls [in Rust]".

embedding-shape•1h ago

It's devbait, not many of us can resist bikeshedding about the title which obviously doesn't accurately reflect the article contents. And the article contents are self-aware enough to admit this to itself too, yet the title remains.

win311fwg•1h ago

The title would suggest that it was already written in Rust; that it was the rewrite in Go that brought five times faster.

IshKebab•1h ago

I vaguely recall that there's a Rust macro to automatically convert recursive functions to iterative.

But I would just increase the stack size limit if it ever becomes a problem. As far as I know the only reason it is so small is because of address space exhaustion which only affects 32-bit systems.

embedding-shape•1h ago

> I vaguely recall that there's a Rust macro to automatically convert recursive functions to iterative.

Isn't that just TCO or similar? Usually a part of the compiler/core of the language itself, AFAIK.

koverstreet•57m ago

I haven't been following become/TCO in Rust - but what I've usually seen is TCO getting flipped off because it interferes with backtraces and debugging.

So I think there's value in providing it as an explicit opt-in; that way when you're reading the code, you know to account for it when you're looking at backtraces.

Additionally, if you're relying on TCO it might be a major bug if the compiler isn't able to apply it - and optimizations that aren't applied are normally invisible. This might mean you could get an error if you're expecting TCO and you or the compiler screwed something up.

jeroenhd•1h ago

Explicit tail call optimization is in the works but I don't think it's available in stable jut yet.

The `become` keyword has already been reserved and work continues to happen (https://github.com/rust-lang/rust/issues/112788). If you enable #![feature(explicit_tail_calls)] you can already use the feature in the nightly compiler: https://play.rust-lang.org/?version=nightly&mode=debug&editi...

(Note that enabling release mode on that link will have the compiler pre-calculate the result so you need to put it to debug mode if you want to see the assembly this generates)

yodacola•1h ago

FlatBuffers are already faster than that. But that's not why we choose Protobuf. It's because a megacorp maintains it.

nindalf•1h ago

You're saying we choose Protobufs [1] because Google maintains it but not FlatBuffers [2]?

[1] - https://github.com/protocolbuffers/protobuf: Google's data interchange format

[2] - https://github.com/google/flatbuffers: Also maintained by Google

rafaelmn•1h ago

I get the OP is off base with his remark - but at the same time maintained by Google means shit in practice.

AFAIK they have a bunch of production infra on protobuff/gRPC - not so sure about flatbufferrs which came out of the game dev side - that's the difference maker to me - which project is actually rooted in.

dewey•1h ago

> but at the same time maintained by Google means shit in practice.

If you worked on Go projects that import Google protobuf / grpc / Kubernetes client libraries you are often reminded of that fact.

secondcoming•58m ago

Yet they've yet to release their internal optimisation that allows zero-copying string-type fields.

rozenmd•1h ago

"5 times faster" reminds me of Cap'n Proto's claim: in benchmarks, Cap’n Proto is INFINITY TIMES faster than Protocol Buffers: https://capnproto.org/

7777332215•58m ago

In my experience capn proto is much less ergonomic.

gf000•29m ago

I mean, cap'n'proto is written by the same person who created protobuf, so they are legit (and that somewhat jokish claim is simply that it requires no parsing).

t-writescode•1h ago

Just for fun, how often do regular-sized companies that deal in regular-sized traffic need Protobuf to accomplish their goals in the first place, compared to JSON or even XML with basic string marshalling?

tcfhgj•59m ago

Well, protobuf allows to generate easy to use code for parsing defined data and service stubs for many languages and is one of the faster and less bandwidth wasting options

bluGill•45m ago

In most languages protobuf is eaiser because it generates the boilerplate. And protobuf is cross language so even if you are working in javascript where json is native protobuf is still faster because the other side can be whatever and you are not spending their time parsing.

jonathanstrange•14m ago

Protobuf is fantastic because it separates the definition from the language. When you make changes, you recompile your definitions to native code and you can be sure it will stay compatible with other languages and implementations.

Chiron1991•5m ago

It's not just about traffic. IoT devices (or any other low-powered devices for that matter) also like protobuf because of its comparatively high efficiency.

steeve•59m ago

tldr: they replaced using protobuf as the type system across language boundaries for FFI with true FFI

Xunjin•47m ago

I loved, every clickbait title should come with a tldr just like this one.

lowdownbutter•52m ago

Don't read clickbaity headlines and scan hacker news five times faster.

Ghostty's AI Policy

I built a light that reacts to radio waves [video]

AI Is a Horse (2024)

Replacing Protobuf with Rust to go 5 times faster

Proton Spam and the AI Consent Problem

Show HN: isometric.nyc – giant isometric pixel art map of NYC

GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

Capital One to acquire Brex for $5.15B

The State of Modern AI Text to Speech Systems for Screen Reader Users

Why does SSH send 100 packets per keystroke?

I was banned from Claude for scaffolding a Claude.md file?

Qwen3-TTS family is now open sourced: Voice design, clone, and generation

TI-99/4A: Leaning More on the Firmware

Douglas Adams on the English–American cultural divide over "heroes"

Google is ending full-web search for niche search engines

Your app subscription is now my weekend project

Bugs Apple Loves

Our collective obsession with boredom: Interview with a boredom lab researcher

Scaling PostgreSQL to power 800M ChatGPT users

Improving the usability of C libraries in Swift

Writing First, Tooling Second

Why medieval city-builder video games are historically inaccurate (2020)

Show HN: Txt2plotter – True centerline vectors from Flux.2 for pen plotters

Project Mercury and the Sofar Bomb

'Askers' vs. 'Guessers' (2010)

Stunnel

CSS Optical Illusions

In Europe, wind and solar overtake fossil fuels

Launch HN: Constellation Space (YC W26) – AI for satellite mission assurance

'Active' sitting is better for brain health: review of studies

Ghostty's AI Policy

I built a light that reacts to radio waves [video]

AI Is a Horse (2024)

Replacing Protobuf with Rust to go 5 times faster

Proton Spam and the AI Consent Problem

Show HN: isometric.nyc – giant isometric pixel art map of NYC

GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers

Capital One to acquire Brex for $5.15B

The State of Modern AI Text to Speech Systems for Screen Reader Users

Why does SSH send 100 packets per keystroke?

I was banned from Claude for scaffolding a Claude.md file?

Qwen3-TTS family is now open sourced: Voice design, clone, and generation

TI-99/4A: Leaning More on the Firmware

Douglas Adams on the English–American cultural divide over "heroes"

Google is ending full-web search for niche search engines

Your app subscription is now my weekend project

Bugs Apple Loves

Our collective obsession with boredom: Interview with a boredom lab researcher

Scaling PostgreSQL to power 800M ChatGPT users

Improving the usability of C libraries in Swift

Writing First, Tooling Second

Why medieval city-builder video games are historically inaccurate (2020)

Show HN: Txt2plotter – True centerline vectors from Flux.2 for pen plotters

Project Mercury and the Sofar Bomb

'Askers' vs. 'Guessers' (2010)

Stunnel

CSS Optical Illusions

In Europe, wind and solar overtake fossil fuels

Launch HN: Constellation Space (YC W26) – AI for satellite mission assurance

'Active' sitting is better for brain health: review of studies

Replacing Protobuf with Rust to go 5 times faster

Comments