I built a message broker that speaks the Kafka wire protocol, so any Kafka
client (librdkafka, kafka-python, kcat, etc.) works without code changes.
The entire binary is 52KB. No JVM, no ZooKeeper, no third-party libraries —
just C++20 with kqueue/epoll. Starts in <10ms, uses 0% CPU when idle.
I built this because running Kafka locally for development is painful —
gigabytes of RAM, slow startup, ZooKeeper/KRaft configuration. I just
wanted something that accepts produce requests and gets out of the way.
Technical details:
- Single-threaded event loop (kqueue on macOS, epoll on Linux)
- Memory-mapped log segments (1GB pre-allocated, sequential I/O)
- Lock-free SPSC/MPSC ring buffers with cache-line alignment
- Kafka protocol v0-v3 including flexible versions (ApiVersions, Metadata, Produce)
- Auto-topic creation on first produce or metadata request
The most interesting bug I hit: librdkafka sends ApiVersions v3, which uses
Kafka's "flexible versions" encoding. But there's a special exception in the
protocol — ApiVersions responses must NOT include header tagged_fields for
backwards compatibility. One extra byte shifted every subsequent field,
causing librdkafka to compute a ~34GB malloc that crashed immediately.
Current limitations: no consumer groups, no replication, single-threaded,
no auth. It's v0.1.0 — consume support is next.
MIT licensed, runs on macOS (Apple Silicon + Intel) and Linux.