Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•11mo ago

Comments

nine_k•11mo ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•11mo ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•11mo ago

They mention this in the article.

spockz•11mo ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•11mo ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•11mo ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•11mo ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•11mo ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•11mo ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•11mo ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•11mo ago

This was very well written. Excellent article!

NetOpWibby•11mo ago

Is this like MessagePack for Haskell?

Startups Are Context Arbitrages

Coq theorem prover is now called Rocq

Sage AI – Specialized AI agents that execute workflows

Printing real headline news on the Commodore 64 with The Newsroom's Wire Service

Space Force looks at moving "significant number" of launches from ULA to SpaceX

Opting out of cookies no guarantee

How Accurate Are Google's A.I. Overviews?

Lowdefy v5: The Config Webstack

I made Agentation for vanilla JavaScript

Project Glasswing Has a Blind Spot. It's You

Users lose $9.5M to fake Ledger wallet app on the Apple App Store

Jane Street Signs $6B AI Cloud Agreement with CoreWeave

Linux 7.1 Is a Big Win for Intel Panther Lake with Fred Now Enabled by Default

Keyword Scout

ChatGPT, Is This Real?

Show HN: EmbedIQ – Claude Code Compliance Config for HIPAA/PCI-DSS/SOC2

We Built Hanker in 14 Days with Claude

Fiverr Denies Report of Data Leak

AI papers published in 2026 worth reading

Claude Cowork found me a flat to rent in London in just 5 days

Project Maven Put A.I. Into the Kill Chain

How China is wooing Paraguay's political class away from longtime ally Taiwan

The Courage to Stop

Anthropic's rise is giving some OpenAI investors second thoughts

Study of the cosmos proves we still can't explain how the universe is expanding

AAUP does not want you to share your syllabus

Show HN: Horizontally Scale Localhost

Before he wrote AI 2027, he predicted the world in 2026. How did he do?

Shoe brand Allbirds says it will become an AI company, sending shares soaring

Generating a Color Spectrum for an Image