Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•8mo ago

Comments

nine_k•8mo ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•8mo ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•8mo ago

They mention this in the article.

spockz•8mo ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•8mo ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•8mo ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•8mo ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•8mo ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•8mo ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•8mo ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•8mo ago

This was very well written. Excellent article!

NetOpWibby•8mo ago

Is this like MessagePack for Haskell?

Surpassing Transformer Efficiency via Singular Geometric Strikes

Drivers divided on 'American Autobahn' plan to pull speed limits of highways

Using an expensive model made our agent 75% cheaper

Notes and Hacks on Germany's Exit Tax

Pentagon moves to cut U.S. participation in some NATO groups

Amazon EC2 G7e Instances Accelerated by RTX Pro 6000 Blackwell GPUs

Cow Uses Broom as a Tool

Netflix earned $1.5B from ads in 2025

Ordered from Newest to Oldest

Hackers

Ask HN: Why are so many rolling out their own AI/LLM agent sandboxing solution?

Show HN: I figured out how to get consistent UI from Claude Code

One Killed in train derailment near Barcelona days after deadly rail collision

Show HN: Date Clue – I built a modern version of magazine dating quizzes

Plastic Waste to Pharmaceuticas: Upcycling Through Ruthenium Semi-Hydrogenation

Ask HN: What have you built/shipped with Claude-code

Digital Omnibus Report V2: Analysis of Select GDPR and EPrivacy Proposals by EC

Penn Calls Government's Demand for Lists of Jewish Staff 'Disconcerting'

macOS Stats: Local Privilege Escalation via Exposed XPC Method

Zero to One: Learning Agents and Agentic Patterns

In a warming world, freshwater production is moving deep beneath the sea

Skyreader: A RSS Reader on the AT Protocol

Being creative requires taking risks

Hacker Lists Vibecoded Apps: 198 Scanned, 196 Found Vulnerable

Spotlight Rules

Claude Chill: Fix Claude Code's Flickering in Terminal

The Surprising Way AI Models Are Helping Humans Communicate Better

How to generate 50K token documents using an agentic scaffold

FastMCP 3.0: From Tool Servers to Context Applications

Show HN: Free interactive security awareness library