Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•8mo ago

Comments

nine_k•8mo ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•8mo ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•8mo ago

They mention this in the article.

spockz•8mo ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•7mo ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•8mo ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•8mo ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•8mo ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•8mo ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•8mo ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•8mo ago

This was very well written. Excellent article!

NetOpWibby•8mo ago

Is this like MessagePack for Haskell?

How I Make These ASCII Pictures and Links to Other Tutorials (2000)

Agentic Coding Course

The Offline Society

Introduction to Agents

PEP 686 – Make UTF-8 mode default

New Way You Can Discover Asteroids

Keeping Windows and macOS alive past their sell-by date: Part 1

Show HN: Why many AI-generated websites don't show up on Google

Sina Hartung Ousted as the CEO of CodeRabbit

Ask HN: What is the international distribution/statistics of HN visitors?

Leaving Meta as Engineering Manager after 6.5 years

EngineAI's T800 humanoid: agile motion, 29-DOF joints and combat-style demos

Show HN: Bookmarklet shows local- and sessionStorage. e.g. on mobile browser

New image sensor breaks optical limits

Why FedRAMP Authorization and CMMC Level 2 Are Now Table Stakes for GovCon AI

Show HN: Another Voice dictation and voice-to-prompt for macOS

Why most software fails solar reps

I Sell Onions on the Internet

Vibesbench: A Multi-Turn Conversational AI Benchmark

Voice Agents Now Retrieve Context in <150ms with On-Device Prediction

Show HN: Buoy – A persistent, status-bar web server for local utilities

Marissa Mayer's new startup Dazzle raises $8M led by Forerunner's Kirsten Green

Algerian law declares France's colonisation a crime

I just built an online tool to RBF Bitcoin transactions

Earth needs more energy: Atlanta's Super Soaker creator may have a solution

You may be able to change your old, embarrassing Gmail address

Evolution and Scale of Uber's Delivery Search Platform

Ask HN: What skills do you want to develop or improve in 2026?

Show HN: Pocket8 – Native PICO-8 Runtime for iOS (WASM to Swift Bridge)

Ask HN: Why is savannah.gnu.org down so much and what are the alternatives?