Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•8mo ago

Comments

nine_k•8mo ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•8mo ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•8mo ago

They mention this in the article.

spockz•8mo ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•8mo ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•8mo ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•8mo ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•8mo ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•8mo ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•8mo ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•8mo ago

This was very well written. Excellent article!

NetOpWibby•8mo ago

Is this like MessagePack for Haskell?

Show HN: Securing Docker Builds

Show HN: AI in SolidWorks

Show HN: Woid – 3x Faster Runtime Polymorphism. C++23

Chesspire: "Slay the Spire" but for Chess

What If ASI Leads to Stasis?

Enterprise Integration Patterns: Process Manager

Pwning Claude Code in 8 Different Ways

Cursor vs. antigravity after a week of real use

Show HN: Senlo - self-hosted open-source email management system

The truth behind the 2026 J.P. Morgan Healthcare Conference

PutHouse – Earn income automatically with risk management built-in

Apple Foundation Models will be based on Gemini

Deft: A new replacement for Clojure objects using plain maps

Framework: Memory and Storage Pricing Updates

I spent my winter break teaching an LLM to play Diplomacy with RL

Forget about crop diseases and try this

Yellowstone Bison Herd

Show HN: Image0.dev – image tools that run in the browser

Telegram recovery model allows permanent lockout after phishing

1X World Model – From Video to Action: A New Way Robots Learn

Apple picks Google's Gemini AI for its big Siri upgrade

All the rovers heading to the Moon over the next 10 years

Mac OLM File to PST Converter"

iPhone 4 is having a TikTok revival

They Want You to "Quit Demonstrating"

In Defense of the New York City Transit Strike

Pi Monorepo: AI agent toolkit

How to Build a Habit

Show HN: Java In-Memory search using Forage

Kavia AI now supports Bitbucket (agent-driven code analysis and regression diff)