Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•11mo ago

Comments

nine_k•11mo ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•11mo ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•11mo ago

They mention this in the article.

spockz•11mo ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•10mo ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•11mo ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•11mo ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•11mo ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•11mo ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•11mo ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•11mo ago

This was very well written. Excellent article!

NetOpWibby•11mo ago

Is this like MessagePack for Haskell?

Weaponizing Extension Packs with PackRAT

LA Jurors Find Meta, YouTube Liable for Youth Mental Health Problems, Award $6M

Verdicts against social media companies carry consequences. But questions linger

VS Code plugin to annotate in Markdown previews for your AI

Web and File Server Comparison (NT Server 4.0 versus RedHat 5.2 on Linux 2.2.2)

Isolated Web Apps

Every Online Conversation about AI, Summarized

KKR to acquire Nothing Bundt Cakes for over $2B

Docker Coding Agent Sandboxes (2025)

Your First Parser

Fyrox Game Engine 1.0.0

Buy Them the Tools

Electric Motorcycles Don't Have to Be Security Nightmares, but This One Was

SpaceX set to file for share sale

FCC imposes ban on new foreign-made routers, allows 2 yrs firmware updates

Mark to Market

Background Agents Landscape

MCP Server for ERPNext / Frappe ERP – MCP Apps

Ask HN: Why is a "War ON Iran" being labelled as a "War IN Iran"?

streaming service for reading (6M+ titles)

The Alignment Illusion

Autoresearch Is Reward Function Design

Show HN: Got fed up missing cool AI events: built tool to match them to my goals

The Corporate Bullshit Receptivity Scale

LiteLLM Has Been Compromised, Do Not Update

Show HN: AgentVerse – Open social network for AI agents (Mar 2026)

'Space Archaeology' Reveals First Dynamic History of a Giant Spiral Galaxy

Show HN: Is Claude Dumb Today?

Show HN: Does anyone still read books?

Emotional Eating Elucidated in Fruitfly