Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•1y ago

Comments

nine_k•1y ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•1y ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•1y ago

They mention this in the article.

spockz•1y ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•1y ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•1y ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•1y ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•1y ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•1y ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•1y ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•1y ago

This was very well written. Excellent article!

NetOpWibby•1y ago

Is this like MessagePack for Haskell?

Show HN: Legant gives AI agents bounded authority to act on your behalf

I don't believe, I know (2019)

Google's online dominance is showing signs of cracking in AI era

Incident Fest 2026

Show HN: Free tools to audit a landing page and check if AI recommends you

Free EU AI Act Article 50 compliance checker for indie SaaS

Seawater batteries for energy storage, desalination and carbon sequestration

Could less caffeine be smarter performance enhancer? Scientists find sweet spot

How to get GLM 5.2 to 280 tokens per second

ByteDance's Seedance 2.5 breaks the 30-second barrier for AI video generation

Boogu-Image – open-source image generation and editing model

Skillhealth: Find Claude Code skills you never use and what they cost

Two Months In: Assessing the Impact of NIST's Enrichment Cutbacks

Show HN: Screen recording app with automatic zoom, subtitles, and more (macOS)

How the Space Program Has Influenced What We Carry

Nub – A Bun-like toolkit that extends Node.js instead of trying to replace it

LiteLLM Migrates to Rust

Faster aging in younger generations linked to rise in early-onset cancer

Bluerails Discovery

The AI Last Mile Is the First Mile

Your AI Agent Should Go to Therapy

Systemd-Bsod.service

Verity.md – Gates, Memory and Cost control for coding agents

With Starfall, SpaceX eyes an edge in global cargo delivery from orbit

Show HN: Cc-fleet – run other LLMs as Claude Code workers, your sub drives

Lost memoir of Hiroshima survivor found after decades in US archive

Show HN: AGI will not come from better models, it will come from meta harness

A simple physics-inspired model sheds light on how AI learns

MSG Made Dossier on Activists Who Opposed Facial Recognition

AI and Math State of the Art