Longhorn – A Kubernetes-Native Filesystem

https://vegard.blog.engen.priv.no/?p=518

31•jandeboevrie•3d ago

Comments

d3Xt3r•3d ago

Longhorn was the codename for Windows Vista... so not a great choice of a name (IMO).

onionisafruit•3h ago

Longhorn is a fine name, and it doesn't matter if somebody else used it 20+ years ago

fineallaround•2h ago

As a codename, no less. 0-0

What a stupid thing to complain about.

privatelypublic•2h ago

Even complaining about Vista raises eyebrows. It had two huge issues: overactive UAC, and Microsoft handing "Vista Certified" to basically anybody who asked. (Frequently to machines that would barely run XP pre-SP1.)

Most of the complaints can be reduced to one of those.

Yes- I hand wave away a lot of other things: because they were required for a huge step towards a decently secure and stable OS.

weinzierl•2h ago

By that logic Titanic would be a fine name too.

NewJazz•1h ago

Hmm, maybe just shorten to Titan?

bigstrat2003•1h ago

I mean, I think it would be. Superstition about naming is silly.

tracker1•2h ago

I remembered the Windows Vista reference as soon as I saw the name. That said, I don't think it's a big deal.

gdbsjjdn•1h ago

I did this was going to be about the Vista and how some of the FS stuff that got cut was prescient. "This old thing that didn't work was ahead of its' time" is a whole genre of post (ex. Itanium)

dilyevsky•2h ago

Anyone knows what's the story with NVMEoF/SPDK support these days? A couple years ago Mayastor/OpenEBS was running laps around Longhorn on every performance metrics big time, not sure if anything changed there...

coopreme•2h ago

Go with Ceph… a little more of a learning curve but overall better.

cmeacham98•2h ago

I tried longhorn on my homelab cluster. I'll admit it's possible that I did something wrong, but I managed to somehow get it into a state where it seemed my volumes got permanently corrupted. At the very least I couldn't figure out how to get my volumes working again.

When restoring from backup I went with Rook (which is a wrapper on ceph) instead and it's been much more stable, even able to recover (albeit with some manual intervention needed) from a total node hardware failure.

nerdjon•17m ago

It is interesting seeing this article come up since just yesterday I setup longhorn in my homelab cluster needing better performance for some tasks than NFS was providing so I setup a raid on my r630 and tried it out.

So far things are running well but I can't shake this fear that I am in for a rude awakening and I loose everything. I backups but the recovery will be painful if I have to do it.

I will have to take a look at rook since I am not quite committed enough yet (only moved over 2 things) to switch.

positisop•2h ago

Longhorn is a poorly implemented distributed storage layer. You are better off with Ceph.

yupyupyups•2h ago

I've heard Ceph is expensive to run. But maybe that's not true?

jauntywundrkind•1h ago

I'm only just wading in, after years of intent. I don't feel like Ceph is particularly demanding. It does want a decent amount of ram. 1GB each for monitor, manager, and metadata, up to 16GB total for larger clusters, according to docs. But then each disk's OSD defaults to 4gb, which can add up fast!! And some users can use more. 10Gbe is recommended and more is better here but that seems not unique to ceph: syncing storage will want bandwidth. https://docs.ceph.com/en/octopus/start/hardware-recommendati...

westurner•1h ago

This from 2023 says: https://www.redhat.com/en/blog/ceph-cluster-single-machine :

> All you need is a machine, virtual or physical, with two CPU cores, 4GB RAM, and at least two or three disks (plus one disk for the operating system).

xyzzy123•1h ago

For me it was the ram for the OSDs, 1GB per 1TB but ideally more for SSDs...

keeperofdakeys•1h ago

Ceph overheads aren't that large for a small cluster, but they grow as you add more hosts, drives, and more storage. Probably the main gotcha is that you're (ideally) writing your data three times on different machines, which is going to lead to a large overhead compared with local storage.

Most resource requirements for Ceph assume you're going for a decently sized cluster, not something homelab sized.

willbeddow•1h ago

have not used longhorn, but we are currently in the process of migrating off of ceph after an extremely painful relationship with it. Ceph has fundamental design flaws (like the way it handles subtree pinning) that, IMO, make more modern distributed filesystems very useful. SeaweedFS is also cool, and for high performance use cases, weka is expensive but good.

q3k•1h ago

That sounds more like a CephFS issue than a Ceph issue.

(a lot of us distrust distributed 'POSIX-like' filesystems for good reasons)

__turbobrew__•32m ago

Are there any distributed POSIX filesystems which don’t suck? I think part of the issue is that POSIX compliant filesystem just doesn’t scale, and you are just seeing that?

willbeddow•14m ago

weka seems to Just Work from our tests so far, even under pretty extreme load with hundreds of mounts on different machines, lots of small files, etc... Unfortunately it's ungodly expensive.

studmuffin650•51m ago

Where I work, we primarily use Ceph for the a K8s Native Filesystem. Though we still use OpenEBS for block store and are actively watching OpenEBS mayastor

__turbobrew__•35m ago

I looked into mayastor and the NVME-of stuff is interesting, but it is so so so far behind ceph when it comes to stability and features. One ceph has the next generation crimson OSD with seastore I believe it should close a lot of the performance gaps with ceph.

scubbo•13m ago

(Copied from[0] when this was posted to lobste.rs) Longhorn was nothing but trouble for me. Issues with mount paths, uneven allocation of volumes, orphaned undeletable data taking up space. It’s entirely possible that this was a skill issue, but still - never touching it again. Democratic-csi[1] has been a breath of fresh air by comparison.

[0] https://lobste.rs/s/vmardk/longhorn_kubernetes_native_filesy... [1] https://github.com/democratic-csi/democratic-csi

KDE launches its own distribution

ChatGPT Developer Mode: Full MCP client access

Show HN: Term.everything – Run any GUI app in the terminal

DOOMscrolling: The Game

Intel's E2200 "Mount Morgan" IPU at Hot Chips 2025

Pontevedra, Spain declares its entire urban area a "reduced traffic zone"

Fraudulent Publishing in the Mathematical Sciences

Launch HN: Recall.ai (YC W20) – API for meeting recordings and transcripts

A polyglot's guide to multiple-dispatch

The HackberryPi CM5 handheld computer

Defeating Nondeterminism in LLM Inference

OrioleDB Patent: now freely available to the Postgres community

Longhorn – A Kubernetes-Native Filesystem

Mux (YC W16) Is Hiring Engineering ICs and Managers

Jiratui – A Textual UI for interacting with Atlassian Jira from your shell

Clojure's Solutions to the Expression Problem

Dotter: Dotfile manager and templater written in Rust

Show HN: Haystack – Review pull requests like you wrote them yourself

Picat: A Logic-based Multi-paradigm Language (2014) [pdf]

I didn't bring my son to a museum to look at screens

"No Tax on Tips" Includes Digital Creators, Too

A desktop environment without graphics (tmux-like)

Harvey Mudd Miniature Machine

Show HN: TailGuard – Bridge your WireGuard router into Tailscale via a container

Charlie Kirk killed at event in Utah

Kerberoasting

Things you can do with a debugger but not with print debugging

In 1979 one of the best guitar solos recorded was cut for radio time

Tarsnap is cozy

Semantic Line Breaks (2017)

Longhorn – A Kubernetes-Native Filesystem

Comments

KDE launches its own distribution

ChatGPT Developer Mode: Full MCP client access

Show HN: Term.everything – Run any GUI app in the terminal

DOOMscrolling: The Game

Intel's E2200 "Mount Morgan" IPU at Hot Chips 2025

Pontevedra, Spain declares its entire urban area a "reduced traffic zone"

Fraudulent Publishing in the Mathematical Sciences

Launch HN: Recall.ai (YC W20) – API for meeting recordings and transcripts

A polyglot's guide to multiple-dispatch

The HackberryPi CM5 handheld computer

Defeating Nondeterminism in LLM Inference

OrioleDB Patent: now freely available to the Postgres community

Longhorn – A Kubernetes-Native Filesystem

Mux (YC W16) Is Hiring Engineering ICs and Managers

Jiratui – A Textual UI for interacting with Atlassian Jira from your shell

Clojure's Solutions to the Expression Problem

Dotter: Dotfile manager and templater written in Rust

Show HN: Haystack – Review pull requests like you wrote them yourself

Picat: A Logic-based Multi-paradigm Language (2014) [pdf]

I didn't bring my son to a museum to look at screens

"No Tax on Tips" Includes Digital Creators, Too

A desktop environment without graphics (tmux-like)

Harvey Mudd Miniature Machine

Show HN: TailGuard – Bridge your WireGuard router into Tailscale via a container

Charlie Kirk killed at event in Utah

Kerberoasting

Things you can do with a debugger but not with print debugging

In 1979 one of the best guitar solos recorded was cut for radio time

Tarsnap is cozy

Semantic Line Breaks (2017)