Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•1y ago

Comments

nine_k•1y ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•1y ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•1y ago

They mention this in the article.

spockz•1y ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•1y ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•1y ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•1y ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•1y ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•1y ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•1y ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•1y ago

This was very well written. Excellent article!

NetOpWibby•1y ago

Is this like MessagePack for Haskell?

Voxl: Fast remote desktop streaming written in Rust

ToothPaste

Apenwarr.ca/thundersnap: an undo button for everything

Would Local AI Become the Dr.Pepper?

Getting hit with "Prove you are a human" at every other website

Bambu Lab H3C - 9 nozzle printer

The Death of the Angel Investor

Show HN: Every book, film and game HN has recommended since 2006

Death Comes for XSLT

Critically endangered pygmy raccoons observed making their own paper toys

Finite Agent Machines: guarantees for dynamic agent swarms [pdf]

How Our AI Agents Write a Help Center

StateBench: Testing how language models track state changes over long documents

Democrats run more anti-tech ads amid AI boom

Show HN: Scroll from your own skeleton to the edge of the observable universe

The Inference Stack Behind OCRia

I use AI (Oct 2026)

Show HN: DrawDB – turn any live database into an ER diagram with one query

Leona - AI Powered Career Toolkit

Apple's New iPhone Release Schedule Seems to Pay Off

Counted "AI" on 37 website builder homepages, 2019 to now

Show HN: AI Model Token Cost and Context Window Estimator

Fedora 45 Released

Are you checking keyword ranking manually?

Agentic Cheminformatics with Google Antigravity

A New Map of 'Empty' Space Challenges a Tranquil Picture of the Galaxy

Itch.io deletes terabytes of all the AI game and asset slop spam

Bugrewind

Show HN: FinZot – financial planning Claude connector for US taxpayers

192.0.64.0 – 192.0.127.255 IP Range