Packed Data Support in Haskell

https://arthi-chaud.github.io/posts/packed/

77•matt_d•1y ago

Comments

nine_k•1y ago

> Introducing the ‘packed’ data format, a binary format that allows using data as it is, without the need for a deserialisation step. A notable perk of this format is that traversals on packed trees is proven to be faster than on ‘unpacked’ trees: as the fields of data structures are inlines, there are no pointer jumps, thus making the most of the L1 cache.

That is, a "memory dump -> zero-copy memory read" of a subgraph of Haskell objects, allowing to pass such trees / subgraphs directly over a network. Slightly reminiscent of Cap'n Proto.

90s_dev•1y ago

We are always reinventing wheels. If we didn't, they'd all still be made of wood.

Zolomon•1y ago

They mention this in the article.

spockz•1y ago

It reminds me more of flat buffers though. Does protobuf also have zero allocation (beyond initial ingestion) and no pointer jumps?

cstrahan•1y ago

No, one example of why being variable sized integers.

See https://protobuf.dev/programming-guides/encoding/

carterschonwald•1y ago

One thing that sometimes gets tricky in these things is handling Sub term sharing. I wonder how they implemented it.

tlb•1y ago

> the serialised version of the data is usually bigger than its in-memory representation

I don’t think this is common. Perhaps for arrays of floats serialized as JSON or something. But I can’t think of a case where binary serialization is bigger. Data types like maps are necessarily larger in memory to support fast lookup and mutability.

nine_k•1y ago

I suppose all self-describing formats, like protobuf, or thrift or, well, JSON are bigger than the efficient machine representation, because they carry the schema in every message, one way or another.

IsTom•1y ago

If you use a lot of sharing in immutable data it can grow a lot when serializing. A simple pathological example would be a tree that has all left subtrees same as the right ones. It takes O(height) space in memory, but O(2^height) when serialized.

gitroom•1y ago

honestly i wish more stuff worked this way - fewer hops in memory always makes me happy

lordleft•1y ago

This was very well written. Excellent article!

NetOpWibby•1y ago

Is this like MessagePack for Haskell?

AI 'productivity pollution': the hidden tax on everyone

Maintaining an organizational knowledge graph with an LLM and event sourcing

Turn Hacker News into a personalized brief

Show HN: Claude Code kept ignoring our MCP tools, so we used hooks instead

A few strong opinions about AI

Researcher Launches 'CPU Deoptimization' Project to Identify Slowest x86

SpaceXAI's Grok 4.6 Scores 61 on the Artificial Analysis Intelligence Index

Can AI do novel security research? Meet the HTTP Terminator

We tested six $24 Linode VMs in Los Angeles: CPU stayed flat, storage varied 76%

HTML over WebSockets: real-time SPAs with barely any JavaScript

FinCEN Ends Beneficial Ownership Reporting for Small Business Owners

I "wrote" my own 8086 nostalgia emulator

Four Lads Who Shook the (Old) World: Alessio, Andy, Jervis, & Tuomas Drop By

Show HN: Snipsy – turn photos into a dated stamp keepsake, sent the same day

Live: Total Solar Eclipse – August 12, 2026 [Live Stream] [video]

Wisconsin Primary Election Results

In case you think there is consciousness, intelligence or personality in an LLM [video]

New York State Newspapers on Microfilm

Kalshi and Polymarket bets on clinical trials criticized as 'ghastly'

Beijing is forcing a mass breakup with AI lovers

Using math for solving real-life problems

Who is Anthropic's auditor – and why should we care?

Show HN: Harbofly, reclaim the 20–100GB of dev caches on your Mac

Ada 83/TLALOC: a modern Ada 83 compiler with DIANA 86 HLIR and fasmg back end

Multi-tenant BYOK encryption in PostgreSQL with pgcrypto

Data Center Foes' Newest Tack: Sue Locals over Approval Process

Oracle has drawn up plans for a new round of layoffs this month

Optimizing SQLite for Servers

Glaciers on the Climate Dashboard

DeepSeek V4 Pro 0813 quietly released