Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•1y ago

Comments

nine_k•1y ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•1y ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•1y ago

They mention this in the article.

spockz•1y ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•1y ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•1y ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•1y ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•1y ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•1y ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•1y ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•1y ago

This was very well written. Excellent article!

NetOpWibby•1y ago

Is this like MessagePack for Haskell?

Whats Hacker News About

Misure di sicurezza rafforzate NIS: stato e preparazione

Mamdani Makes a Splash with His Bidets

Sam Altman: 'I No Longer Think We're Gonna Be Able to Just Patch All the Bugs'

Hardwood - modern lightweight Java reader and writer for Parquet file format

The Colors of a Soap Film

How close can pure functional programming get to C?

Show HN: A map of outdoor gyms and pull-up bars

Show HN: Inference Cooperative, user-governed AI

CowAgent – Open-Source Super AI Assistant

The Anger Machine (Joan Kirner Social Justice Oration 2026)

Show HN: Web Bot Auth SDK and SSH Utilty

Show HN: CallFlow – VS Code REST client that also captures your terminal's HTTP

Ruliobiology and the Theoretical Foundations of Biological Form

Black October

Build your coding orchestrator from scratch part 1

Ask HN: How do people self host email and maintain deliverability?

EU Member States approve "digital expropriation" for AI companies

Agent skill for creating isometric illustrations

Meta says if its AI makes a mistake, it will legally be the customer's fault

Modular MAX 26.7 open on October 22 2026

Tabletop Whale is an original science illustration blog

Takeda Hideo credits Ronald Searle, not Yakuza, for his absurdist art

How Block Blast harvests its players for ad money

OpenAI disables internet access during model training

Apple Security Research Device

Anthropic employee tells students their careers may not exist by graduation

Google changes privacy policy so Gmail, Photos could fuel targeted ads

Security incident: Hostwinds client area offline

Half‑Life: Alyx on Apple Vision Pro