Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•8mo ago

Comments

nine_k•8mo ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•8mo ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•8mo ago

They mention this in the article.

spockz•8mo ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•8mo ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•8mo ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•8mo ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•8mo ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•8mo ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•8mo ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•8mo ago

This was very well written. Excellent article!

NetOpWibby•8mo ago

Is this like MessagePack for Haskell?

Encrypt Your .env in a Meme

Communication as a Product

Key US Power Grid [PJM] Cuts Demand Outlook on Overstated AI Boom

Social media time does not increase teenagers' mental health problems – study

The Inelastic Markets Hypothesis [pdf]

The Single-Click Microsoft Copilot Attack That Silently Steals Personal Data

DeepSeek's technical papers show frontier innovation

"Don't worry. Boys are hard to find." Trump/Epstein and... Criminal Enterprises

We built a browser with GPT-5.2 in Cursor

Show HN: Commosta – marketplace to share computing resources

JSON Render

Show HN: IMSAI/Altair inspired microcomputer with web emulator

Skillshare: Sync skills to all your AI CLI tools with one command

Show HN: Chklst – A Minimalist Checklist

Opinion: Why tech leaders can't regulate AI before releasing them?

Vibe Coding Paradox

Show HN: I built a satellite forensic engine to detect fraud in Carbon Markets

Google is shutting down the Tenor API

Bubblewrap: A nimble way to prevent agents from accessing your .env files

Is passive investment inflating a stockmarket bubble?

I beat Factorio on 1k Floppy disks [video]

ISS astronauts return to Earth early due to illness of crew member

2025 Berggruen Prize Essay Competition Winners

AgentDiscover Scanner – Multi-layer AI agent detection (code, network, K8s eBPF)

Skrillex Releases Kora

Kutt.ai – Free AI Video Generator, Text and Image to Video

Personal Intelligence: Connecting Gemini to Google Apps

Mapping Nostr keys to DNS-based internet identifiers

WAPlus' Guide to WhatsApp CRM

Verizon Is Down