Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•1y ago

Comments

nine_k•1y ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•1y ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•1y ago

They mention this in the article.

spockz•1y ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•1y ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•1y ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•1y ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•1y ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•1y ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•1y ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•1y ago

This was very well written. Excellent article!

NetOpWibby•1y ago

Is this like MessagePack for Haskell?

SaaS Cheat Code

Jinn – a Rust microkernel with capability-base security looking f contributors

One photographer has spent two decades building a visual ark of animal life

Show HN: Moching – AI desktop agent with 219 built-in tools (Rust)

Something Interesting

AWS Service Availability Updates

Sculptures honoring N. Korean soldiers to be erected in Vietnam

Basecamp 5 and Hey Outage Postmortem

Strange Parodies of Atari 2600 Video Game Box Cover Art (2008)

Halo CE in the Browser

The Ecology of Memory: Scent, Culture, and the Brain

Remembering the Future: Blade Runner

Let's talk about trading compute

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

MLB considers Grand Teton National Park for baseball game site

Beyond LLMs: A Post-Transformer World Emerges

Show HN: Alter Zero – RAM-efficient agent for coding, security and automation

Show HN: Reflex Engine Beats Both Llama.cpp and vLLM on Cold-Start to TTFT

Important Update About My Apps and Apple Developer Account

Jev-as-a-Judge: Accept When Confident, Escalate When Unsure

How do you deal with people who strongly disagree with your views?

Before Comparing Meta Muse Sentinel,Read the Earlier Disclosures in June 2026

Show HN: Momentum Mouse – slide your phone on a desk and the PC cursor follows

Mark

Want to protect yourself against cancer? Get vaccinated

Thinking out loud AI and Cancer Cure

Why the Bronze Age Collapsed

Diffsmith AI Code Review Tool

Batcher's Bitonic Merger Sort

LabBench: Can AI agents decide what experiment to run next?