Hey HN!

I've built clipper.dev - the fastest container repository in the world. (Big claim, I know.) It deduplicates at a file/chunk level, rather than at the layer level.

I get around 10-20% faster pulls than docker (when using containerd for both) for fresh pulls. For some workflows, I see 80-90% faster patch pulls (think v0.17.5 -> 0.17.6).

Here's a best case scenario for me (and worst case for dockerhub)

    docker> time sudo nerdctl pull ollama/ollama:0.17.5
    docker.io/ollama/ollama:0.17.5:
    elapsed: 244.1s total:  2.9 Gi (12.3 MiB/s)                                      

    real 4m4.172s
    user 0m0.115s
    sys 0m0.227s
    docker> time sudo nerdctl pull ollama/ollama:0.17.6
    docker.io/ollama/ollama:0.17.6:
    elapsed: 234.9s total:  2.8 Gi (12.3 MiB/s)                                      

    real 3m54.965s
    user 0m0.051s
    sys 0m0.275s

---

    clipper> time sudo clipper pull clipper/ollama:0.17.5
    using containerd runtime at /run/containerd/containerd.sock
    fetching manifest clipper.dev/clipper/ollama:0.17.5 ...
    image created: clipper.dev/clipper/ollama:0.17.5
    fetched 46 chunks (2.2 GB, 11.7 MB/s), took 3m16.134s
    done — clipper.dev/clipper/ollama:0.17.5

    real 3m17.532s
    user 0m0.006s
    sys 0m0.012s

    clipper> time sudo clipper pull clipper/ollama:0.17.6
    using containerd runtime at /run/containerd/containerd.sock
    fetching manifest clipper.dev/clipper/ollama:0.17.6 ...
    image created: clipper.dev/clipper/ollama:0.17.6
    fetched 6 chunks (599.2 MB, 7.7 MB/s), took 1m17.48s
    done — clipper.dev/clipper/ollama:0.17.6

    real 1m19.061s
    user 0m0.005s
    sys 0m0.016s

The first two layers of 0.17.6 are shared with 0.17.5, but everything else has to be transported again with docker, but not with clipper. The exact speedup depends on how fast my wifi is feeling at the time, but I'm pulling 1.5 GB less than dockerhub can do. This is _huge_ for robotics workflows or anyone else pulling docker images on poor connections. (This is not even as fast as it could be - there's other optimizations like chunking within files or using BytePort that could make this go even faster).

I built this because at multiple companies I've been at, pulling new images for updates has been...very painful. Robotics Dockerfiles tend to have a lot of dependencies stuffed into a small amount of layers. Adding one new dependency means shipping the bytes for every other dependency, even with perfect pinning. Additionally, the Dockerfiles are oftentimes...not well written, compounding the problem. So I decided to fix the tool to fit the user, rather than the inverse.

There's some other side benefits to using clipper: - Moving a file or changing permissions is "free": another layer already has the bytes - The on-disk structure lends itself much better to P2P device updates than docker's layout

I do intend this to be a commercial product, but pricing is still up in the air. I'm currently targeting robotics companies as they feel the most pain here. There's some other possible fields that I'm interested in that might find use in this: IoT, edge AI, space datacenters/etc, developmer tooling.

Please give it a try. If you don't want to upload an image, these repos should be publically pullable once you register an account:

    clipper/ubuntu: 20.04, 22.04, 24.04
    clipper/ollama: 0.17.5, 0.17.6
    clipper/python: 3.13, 3.14

Fabrica: Autonomous Engineering Plugin for OpenClaw

The Political Disaster That Is California [video][15min]

Amazon Stopping Support for Old Kindles

Show HN: Prefab – A generative UI framework for Python

Local SEO Analyst Agent – PDF Report Generation

Are We in the Chinese Room?

The Hormuz chokehold affects AI funding too

I Don't Care What the Haters Are Saying, I'm Having a Blast

Waymo's Robot Car Testing Ends in NYC After Permits Expire

Digital Assets Rules Need Clarity

U.S. Made a Deal That Gives Us Nothing We Wanted

WireGuard VPN developer's Microsoft account locked

I built a canvas where AI agents work together as a design team

All You Need Is Not All You Need

I built an agent agnostic, locally run, open-source observability product

Wells Fargo whistleblower award slashed by Wall Street watchdog

I built CLI tool that analyzes logs and explains incidents

Show HN: Portable AI memoryswitch providers, keep your context

Peer-to-Peer Republic

Every company is now a media company–and every boss a star

Childhood Mortality by Parental Cause of Death

Can We Measure Software Slop? An Experiment

The genetics of GLP-1 drugs

Show HN: Clipper – The World's Fastest Container Registry

Mama pig and her piglet going outside for the first time after being rescued

Coz: Causal profiling that measures optimization potential

What Are AI Tokens and Context Limits? Why Your AI Forgets What You Told It

Source: An app to prove your photos aren't AI

Career-Ops – multi agent job search

Enjoy Open Source Free Tailwind Component Library – 1,700 Component

Show HN: Git-chain – Simplify branch dependency management in Git

Comments