frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Streaming compression beats framed compression

https://bou.ke/blog/compressed/
14•bouk•3d ago

Comments

lambdaloop•2h ago
Does streaming compression work if some packets are lost or arrive in a different order? Seems like the compression context may end up different on the encoding/decoding side.. or is that handled somehow?
duskwuff•2h ago
It sounds as though the data is being transferred over HTTP, so packet loss/reordering is all handled by TCP.
dgoldstein0•2h ago
I think the underlying protocol would have to guarantee in order delivery - either via tcp (for http1, 2, or spdy), or in http3, within a single stream.
gkbrk•1h ago
WebSockets [1] run over TCP, and the messages are ordered.

There is RFC 9220 [2] that makes WebSockets go over QUIC (which is UDP-based). But that's still expected to expose a stream of bytes to the WebSocket, which still keeps the ordering guarantee.

[1]: https://datatracker.ietf.org/doc/html/rfc6455

[2]: https://datatracker.ietf.org/doc/rfc9220/

duskwuff•2h ago
Before you get too excited, keep two things in mind:

1) Using a single compression context for the whole stream means you have to keep that context active on the client and server while the connection is active. This may have a nontrivial memory cost, especially at high compression levels. (Don't set the compression window any larger than it needs to be!)

2) Using a single context also means that you can't decompress one frame without having read the whole stream that led up to that. This prevents some possible useful optimizations if you're "fanning out" messages to many recipients - if you're compressing each message individually, you can compress it once and send the same compressed message to every recipient.

adzm•1h ago
The analogy to h264 in the original post is very relevant. You can fix some of the downsides by using the equivalent of keyframes, basically. Still a longer context than a single message but able to be broken up for recovery or etc.
efitz•1h ago
When I worked at Microsoft years ago, me and my team (a developer and a tester) built a high volume log collector.

We used a streaming compression format that was originally designed for IBM tape drives.

It was fast as hell and worked really well, and was gentle on CPU and it was easy to control memory usage.

In the early 2000s on a modest 2-proc AMD64 machine we ran out of fast Ethernet way before we felt CPU pressure.

We got hit by the SOAP mafia during Longhorn; we couldn’t convince the web services to adopt it; instead they made us enshittify our “2 bytes length, 2 bytes msgtype, structs-on-the-wire” speed demon with their XML crap.

Google is dead. Where do we go now?

https://www.circusscientist.com/2025/12/29/google-is-dead-where-do-we-go-now/
690•tomjuggler•10h ago•599 comments

GOG is getting acquired by its original co-founder

https://www.gog.com/blog/gog-is-getting-acquired-by-its-original-co-founder-what-it-means-for-you/
607•haunter•14h ago•350 comments

ManusAI Joins Meta

https://manus.im/blog/manus-joins-meta-for-next-era-of-innovation
193•gniting•8h ago•111 comments

Hacking Washing Machines [video]

https://media.ccc.de/v/39c3-hacking-washing-machines
76•clausecker•5h ago•18 comments

Stranger Things creator says turn off "garbage" settings

https://screenrant.com/stranger-things-creator-turn-off-settings-premiere/
119•1970-01-01•7h ago•173 comments

Tesla's 4680 battery supply chain collapses as partner writes down deal by 99%

https://electrek.co/2025/12/29/tesla-4680-battery-supply-chain-collapses-partner-writes-down-dea/
373•coloneltcb•13h ago•401 comments

Show HN: Stop Claude Code from forgetting everything

https://github.com/mutable-state-inc/ensue-skill
129•austinbaggio•8h ago•157 comments

The future of software development is software developers

https://codemanship.wordpress.com/2025/11/25/the-future-of-software-development-is-software-devel...
139•cdrnsf•12h ago•115 comments

Show HN: A Claude Code plugin that catch destructive Git and filesystem commands

https://github.com/kenryu42/claude-code-safety-net
34•kenryu•4d ago•33 comments

AI is forcing us to write good code

https://bits.logic.inc/p/ai-is-forcing-us-to-write-good-code
148•sgk284•12h ago•107 comments

Parsing Advances

https://matklad.github.io/2025/12/28/parsing-advances.html
72•birdculture•7h ago•5 comments

MongoDB Server Security Update, December 2025

https://www.mongodb.com/company/blog/news/mongodb-server-security-update-december-2025
65•plorkyeran•6h ago•19 comments

Outside, Dungeon, Town: Integrating the Three Places in Videogames (2024)

https://keithburgun.net/outside-dungeon-town-integrating-the-three-places-in-videogames/
57•vector_spaces•6h ago•24 comments

Incremental Backups of Gmail Takeouts

https://baecher.dev/stdout/incremental-backups-of-gmail-takeouts/
69•pbhn•4d ago•26 comments

When someone says they hate your product

https://www.getflack.com/p/responding-to-negative-feedback
131•jger15•11h ago•95 comments

Streaming compression beats framed compression

https://bou.ke/blog/compressed/
14•bouk•3d ago•7 comments

Static Allocation with Zig

https://nickmonad.blog/2025/static-allocation-with-zig-kv/
184•todsacerdoti•15h ago•89 comments

100x (YC S22) Is Hiring a Front End Engineer

1•shardullavekar•6h ago

Kidnapped by Deutsche Bahn

https://www.theocharis.dev/blog/kidnapped-by-deutsche-bahn/
996•JeremyTheo•18h ago•878 comments

I migrated to an almost all-EU stack and saved 500€ per year

https://www.zeitgeistofbytes.com/p/bye-bye-big-tech-how-i-migrated-to
120•alexcos•7h ago•71 comments

Geology of the Gulf of the Farallones National Marine Sanctuary

https://pubs.usgs.gov/fs/farallones/
46•greesil•8h ago•13 comments

Karpathy on Programming: “I've never felt this much behind”

https://twitter.com/karpathy/status/2004607146781278521
400•rishabhaiover•3d ago•446 comments

Flame Graphs vs Tree Maps vs Sunburst (2017)

https://www.brendangregg.com/blog/2017-02-06/flamegraphs-vs-treemaps-vs-sunburst.html
116•gudzpoz•2d ago•30 comments

The Signature Flicker

https://steipete.me/posts/2025/signature-flicker
3•tosh•3d ago•0 comments

A production bug that made me care about undefined behavior

https://gaultier.github.io/blog/the_production_bug_that_made_me_care_about_undefined_behavior.html
131•birdculture•13h ago•76 comments

Vitest Browser Mode Guide

https://howtotestfrontend.com/resources/vitest-browser-mode-guide-and-setup-info
44•howToTestFE•5d ago•5 comments

Show HN: Euclidle – Guess the Coordinates in N‑Dimensional Space

https://euclidle.com/
6•bills-appworks•3d ago•3 comments

Linux DAW: Help Linux musicians to quickly and easily find the tools they need

https://linuxdaw.org/
225•prmoustache•18h ago•103 comments

Show HN: A 45x45 Connections Puzzle To Commemorate 2025=45*45

https://thomaswc.com/2025.html
54•thomaswc•6d ago•10 comments

Stanford Lecture: Dr. Don Knuth – Adventures with Knight's Tours [video]

https://www.youtube.com/watch?v=MKiRte-tnMY
58•vismit2000•6d ago•5 comments