Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•1y ago

Comments

nine_k•1y ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•1y ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•1y ago

They mention this in the article.

spockz•1y ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•1y ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•1y ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•1y ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•1y ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•1y ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•1y ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•1y ago

This was very well written. Excellent article!

NetOpWibby•1y ago

Is this like MessagePack for Haskell?

EU high-tech trade surplus up to €32.4B in 2025

Diskwatch: Single-host, read-only disk diagnostics TUI

Show HN: Screen size calculator that puts the TV on your wall in AR, no app

Any recommendation for birth injury lawyer

Show HN: photocn the open-source photo editor for shadcn/ui

Who will physics Nobel 2026?

Show HN: Apple Notes, Photos and Find My as Native Linux Apps (Omarchy/Arch)

Hackers sent push notifications to ASOS users

The Rise of Gig Economy Warfare

Show HN: Fly spacecraft from a Linux shell over an ansible link (browser game)

Ow HN: Macinweb – An OS simulator built in Next.js/Nest.js

Show HN: Mcpward – contract and security testing for MCP servers in CI

Optimizing Weapons Reviews in the Age of Technological Innovation

ASOS app push notification claims hackers breached its customer data warehouse

Libhypercube (find new sounds using audio synthesizers)

ClickHouse Database Management GUI: Tables, SQL, and Schemas

Seagate and Toshiba bid for TDK's HDD head unit amid AI storage boom

Show HN: Use your own Amazon SES account for email workflows

GitHub Slams the Brakes on Private Vulnerability Reports

Treap

Where the Story Starts Working Better Than the System

Copilot.travel – Plan the drive. We'll do the sums

Show HN: Speck – Cognitive architecture for small local LLMs

My New Course at UT Austin: AI Alignment Theory

Technotitlan, 1519 – Experience Aztec's capital in 3D

A comprehensive comparison of the main OSS semantic layers

iPhone Duo display confirmed to peel off on purpose

Let's make /photo pages

A giant list of smart home discounts for prime big deal days (October 6-7,2026)

Show HN: Niquests – Concurrent HTTP in Python, no async or threading code