Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•1y ago

Comments

nine_k•1y ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•1y ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•1y ago

They mention this in the article.

spockz•1y ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•1y ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•1y ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•1y ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•1y ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•1y ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•1y ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•1y ago

This was very well written. Excellent article!

NetOpWibby•1y ago

Is this like MessagePack for Haskell?

RAM crisis provokes enthusiast to try Windows 11 on DDR1-era hardware

What data access pattern is as slow as possible?

Vertical Slices in Practice

Unfathomable bugs #10: The Broken Windows Build

Text Files as a User Interface

When will the decimals in A/B repeat?

Omarchy is the best thing I've stopped thinking about. 1 year review blog

Height of Harmonic Numbers

The Internet Doesn't Need More Content, It Needs More You

Spritework on the ZX Spectrum: Preparing Our Graphics

Life lessons and hot takes from my 30s

Ask HN: Hiii

Google rejects claims of AI in Search hurting web traffic

Ask HN: Sdfs

Ask HN: Are there any developers prefer writing code manually?

Roundup of July 2026 Bootstrapper Events

Fake Windows: highly realistic LED screens that simulates sunlight into a room

The Real Cost of Using AI in 2026

Hungary holds first post-Orban Budapest Pride march

I-Regexp: An Interoperable Regular Expression Format

Can we create a shield to protect Earth from solar storms?

[Humor] You have violated the rules of our community

Itcss: Scalable and Maintainable CSS Architecture (2021)

The US Lock of the Web

Flick – Self-hostable file sharing with human-friendly codes (Go/Next.js)

Show HN: CCA-F Gotchas – exam pitfalls turned into an earworm playlist

Chinese tech hub's shift into robotaxis leaves drivers by the wayside

A Transformer Becomes an LLM

6 Facts About Europe and Air Conditioning

Policymakers can prep for a potential AI job apocalypse