Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•1y ago

Comments

nine_k•1y ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•1y ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•1y ago

They mention this in the article.

spockz•1y ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•1y ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•1y ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•1y ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•1y ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•1y ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•1y ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•1y ago

This was very well written. Excellent article!

NetOpWibby•1y ago

Is this like MessagePack for Haskell?

Escalate hard decisions with the advisor tool

NYT head of engineering and gaming experience allegedly shot by elderly in-laws

Made a thing, if you use Grok and Pi check it out. Or scoll on bros

HouseOS – self-hosted homesystem platform, designed to be operated by agents

Found in translation: the selfless legacy of Eien Ni Hen

Range – open a 1 TB AI model in 3 seconds without downloading it

How I Made a Zero Dependency Terminal Text Editor in C Using POSIX System Calls

Openprinter

Recalld – A memory layer for AI agents that returns only relevant facts

When to choose Sonnet over Opus, what it costs, and how to tune it

Summary Advice (Churchill, 1940)

Walmart promises to never use AI to charge shoppers higher prices

Oil Rises as Strait of Hormuz Talks Stall

Bioenergetic Landscapes

Blade Runner

The AI as the New Compiler

Claude and Jev Play Pokémon

Show HN: What a VM for your AI agent costs across 13 sandbox providers

Show HN: Serverless, cheater-proof MTG engine using zero-knowledge play

ESP32S3 cluster running 1.58-bit (BitNet) Language model

DrawDB: Online database diagram editor and SQL generator

OpenAI, Hugging Face Attack – Music Video

Women Behind North Korea's IT Worker Operations

Remembering the Chicago Pile, the First Nuclear Reactor

Spec Driven Development in 14mins

Cap Snatching

Deutsche Bahn "joke" is no longer funny

Stanford CME295 Transformers n LLMs – Autumn 2026 – Lecture 1 – Transformers [video]

Starship Achieved Orbit

Machine-Generated, Machine-Checked Proofs for a Verified Compiler (ICFP'26) [video]