Show HN: CallFS – S3-style object store in one Go binary (MIT)

66•ebogdum•6mo ago

We started CallFS after yet another late-night “why did the uploads vanish?” incident. Our small team had stitched together rsync, a fragile NFS mount, and an S3 bucket—none of it observable, all of it waiting to bite us.

So we wrote a single-process file service in Go that: • Speaks the S3 API (so existing tooling works). • Stores hot data on local disks for speed; cold data can sit in any S3-compatible bucket. • Exposes Prometheus metrics and JSON logs by default, because “what happened?” shouldn’t be guesswork. • Ships as a ~25 MB static binary—no external deps, MIT license.

Today it’s stable for single-node or side-by-side deployments. Clustering is on the roadmap, replication will follow, but we wanted to share the code early and hear real-world pain points. If storage glue code ever ruined your weekend, we’d love feedback and PRs.

Comments

cowboyscott•6mo ago

I was wanting this to exist today :) Thanks for sharing the work

mt42or•6mo ago

https://github.com/juicedata/juicefs

ebogdum•6mo ago

SO COOL. This is way more than what my scope was, maybe I can learn some stuff.

I do have some future plans on replication and mutex and specific files from specific instances.

Thanks for sharing this.

evil-olive•6mo ago

https://github.com/seaweedfs/seaweedfs is excellent as well

ebogdum•6mo ago

Well, glad to be of service and I really hope it gives you what you need. If not, just open an issue and we will see what we can do. :)

Thanks for sharing your thoughts.

mdaniel•6mo ago

I can't quite square up your "speaks the S3 API" with https://github.com/ebogdum/callfs#file-operations seeming to do its own thing, which for sure would not leverage existing tooling

And it would have been better if in your "static binary" sentence you had included its dependence upon both PostgreSQL and Redis

ebogdum•6mo ago

Very fair points, thank you very much for taking the time. I will make the changes later today to mention the dependencies.

As for the "speaks the S3 API", the idea is that this has two backends right now, Local File System and any S3 compatible API, like AWS S3, MinIO, DigitalOcean Object Storage ... and so on. In other words you can use them all at the same time, provided you have an instance for each.

But you did give me an idea, as to the fact that I need to add some directory/bucket discovery for existing files.

Thank you very much for your feedback.

jasonjmcghee•6mo ago

I had the same question, and started looking more deeply at the project.

Would be interested to hear if this was built by OP or vibe-coded.

ebogdum•6mo ago

A bit of both, as you can imagine, there are so many tedious tasks that consume time for so little in return, so yeah.

moritonal•6mo ago

So you're trying to get buy-in for a tool, when you yourself don't see the point in confirming it works? Those "tedious tasks" are software development.

ebogdum•6mo ago

What, you mean writing documentations and readme?

Of course I'm using this. I have a homelab with 12 PI's, this tool helps me play with files between all of them, any way I want. Might not be the world changing usage you were hopping for, but for me it's enough.

evil-olive•6mo ago

> I have a homelab with 12 PI's, this tool helps me play with files between all of them

did you have your LLM write the Show HN description too?

because you've gone from "We started CallFS" and "our small team" to "I play with it in my homelab"

if this is a homelab-level project, there's absolutely nothing wrong with that. but you should be careful not to mislead people, even unintentionally, about the stability/maturity of the project.

especially when the project involves data storage. vibe-coding a game is one thing, if it has bugs then you might miss a power-up or get stuck on a level with no way out. when a vibe-coded storage system has bugs, you're potentially losing or silently corrupting user data.

ebogdum•6mo ago

You're asking me if I use it, I'm telling you where and how I use it. Then you complain how I use it.

Listen, I can totally respect anyones opinion, but this discussion isn't bringing any benefits, it doesn't seem to be constructive at all.

I get it, you're dissatisfied with this project or whatever, and usually I do try to be very accommodating, as you can see I do engage, but at the same time, please understand, this, is an open-source MIT licensed project, it's not the next "save the earth project" and still, you're acting like you're the VC and already lost money on the investment. Like I have to prove some worth or something.

To answer your question, yeah, there were two of us working at this, for a while, then one lost interest, I thought it would still be fair to say "our small team".

As for the losing data or any other such things, great idea, I will add that there is always a possibility of losing data, just like with NTFS, with AppleFS, or any other FS no matter who developed it or supports it. But it is a good point.

As far as this comment thread goes, I will personally refrain from future comments, as I believe it doesn't serve any good purpose other than nit picking.

Thank you.

MOARDONGZPLZ•6mo ago

They’re clearly not complaining at how you use it, they’re calling out your misrepresentations. These are ethical issues and you should examine your own moral framework.

moritonal•6mo ago

But you brought it here to get feedback? Our feedback is things like, the docs (https://github.com/ebogdum/callfs/blob/main/docs_markdown/02...) have a pile of references to things that just don't exist in your code? You have a change-log that isn't used? Your main file references a site that doesn't exist "https://github.com/ebogdum/callfs/blob/main/cmd/main.go".

MOARDONGZPLZ•6mo ago

Well, to be fair writing documentation is tedious (as are changelogs, and writing code). So they outsourced it to an LLM.

evil-olive•6mo ago

> if this was built by OP or vibe-coded

I hope we start to see some etiquette / conventions developed around this, especially for open source projects.

if I showed up to a dinner party with homemade cookies, my friends would be appreciative.

if I showed up with store-bought cookies, my friends would probably still be appreciative.

but if I showed up with store-bought cookies and tried to claim they were homemade, my friends would probably feel insulted. even if the cookies were amazing, they'd be overshadowed by my dishonesty about them.

the problem, of course, is that if we had some standard section in a readme for expressing the spectrum between "0% LLM output, entirely whittled by hand as a labor of love" vs "100% LLM output, human eyes have never actually looked at any of it" we'd have LLMs hallucinate that readme section, saying they were human-coded.

ebogdum•6mo ago

Will not argue, pro or against, everyone is entitled to their own opinion. But this reminds me of the early 2000 interviews when every one was asking if you used Google search, and now we use "google it" as a verb.

There was a meme at one point with a dude at an interview answering that with something like this: If you google a recipe, it doesn't mean the food is done, you still need someone to cook it.

As for the "xx% LLM" or not, I would argue the same for "xx% Googled", "xx% Stackoverflow" "xx% Learned in school vs copied from the interned", "xx% actual development and engineering vs library importing and glueing".

thomascountz•6mo ago

Likewise, if I show up with homemade cookies, after your friend asks "who are you and why are you in my house," if they then assume my cookies must be store bought and I probably wouldn't be able to have made them myself, I'd think them rather judgmental.

pjc50•6mo ago

Then the more serious one of "do these cookies contain allergen X?"

Any of the answers "yes, no, I don't know" is OK, although the latter is irritating, but if you claim "no" when the allergen actually is present it's a disaster. Whether that's because you bought them and didn't look at the ingredients or made them yourself in a careless shared environment.

For code from the Internet there's the related question "how likely is this to contain security holes or data-loss bugs?" And being LLM-written currently makes that a lot more likely.

techn00•6mo ago

I am so sick of AI generated README's, they follow the samn damn format.

ebogdum•6mo ago

As you can imagine, "story telling" isn't something everyone has mastered, so it does make the job easier. But I do get your point.

Oxodao•6mo ago

We don't want story telling. We just want a clear, few lines long paragraph that tells everything you need to know about the software

dawnerd•6mo ago

I'm so sick of these vibe coded apps. They all end up having security issues. Definitely wouldn't trust it.

ebogdum•6mo ago

You have a very good point.

Definitely, no one would force you to use something you don't find as trustworthy or valuable.

MOARDONGZPLZ•6mo ago

KEY FEATURES :key-emoji:

CONTRIBUTING :handshake-emoji:

CORE FEATURES :apple-core-emoji:

TheDong•6mo ago

> Speaks the S3 API

Let's test that lol.

First, I looked at the docker compose file, seems like an easy way to run it right? Wrong, the docker compose file just runs minio, a real s3 compatible thing: https://github.com/ebogdum/callfs/blob/16e1096095c809f31aa93...

Why check in a docker-compose that only runs someone else's code, not your own project?

Okay, fine, let's run it normally:

    ./callfs server -c ./config.yaml.example
    failed to load config file ./config.yaml.example: file provider does not support this method

Okay, fine, your example config doesn't work with an inscrutable error.

But running it without a config flag works, so let's do that:

    {"level":"info","ts":1752571569.821955,"caller":"cmd/main.go:251","msg":"Starting HTTPS server","addr":":8443"}

Let's see if it's really s3 compatible now:

    AWS_CA_BUNDLE=./server.crt aws --endpoint-url https://localhost:8443 s3 mb s3://foo

    make_bucket failed: s3://foo An error occurred (404) when calling the CreateBucket operation: Not Found

"404" is not the response AWS gives to a make bucket request.

Reading the actual docs, this is _not_ s3 compatible, not even close. So much for "Speaks the S3 API".

Just use minio, or one of the other actually functioning things in this space.

MOARDONGZPLZ•6mo ago

This is a vibe coded project. It doesn’t work very well at all, not going to rehash what others have found, but these projects have been a major plague to the Go community over the last six months. They’re essentially the code version of littering.

sivchari•6mo ago

I'm going to test it :)

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

The Waymo World Model

What Is Ruliology?

How we made geo joins 400× faster with H3 indexes

Unseen Footage of Atari Battlezone Arcade Cabinet Production

Show HN: Look Ma, No Linux: Shell, App Installer, Vi, Cc on ESP32-S3 / BreezyBox

Monty: A minimal, secure Python interpreter written in Rust for use by AI

Show HN: I spent 4 years building a UI design tool with only the features I use

Microsoft open-sources LiteBox, a security-focused library OS

Sheldon Brown's Bicycle Technical Info

Hackers (1995) Animated Experience

Delimited Continuations vs. Lwt for Threads

Show HN: If you lose your memory, how to regain access to your computer?

An Update on Heroku

Dark Alley Mathematics

PC Floppy Copy Protection: Vault Prolok

Was Benoit Mandelbrot a hedgehog or a fox?

Jeffrey Snover: "Welcome to the Room"

How to effectively write quality code with AI

Introducing the Developer Knowledge API and MCP Server

I spent 5 years in DevOps – Solutions engineering gave me what I was missing

Vocal Guide – belt sing without killing yourself

Female Asian Elephant Calf Born at the Smithsonian National Zoo

Why I Joined OpenAI

Understanding Neural Network, Visually

Show HN: R3forth, a ColorForth-inspired language with a tiny VM

I now assume that all ads on Apple news are scams

FORTH? Really!?

Zlob.h 100% POSIX and glibc compatible globbing lib that is faste and better

Learning from context is harder than we thought

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

The Waymo World Model

What Is Ruliology?

How we made geo joins 400× faster with H3 indexes

Unseen Footage of Atari Battlezone Arcade Cabinet Production

Show HN: Look Ma, No Linux: Shell, App Installer, Vi, Cc on ESP32-S3 / BreezyBox

Monty: A minimal, secure Python interpreter written in Rust for use by AI

Show HN: I spent 4 years building a UI design tool with only the features I use

Microsoft open-sources LiteBox, a security-focused library OS

Sheldon Brown's Bicycle Technical Info

Hackers (1995) Animated Experience

Delimited Continuations vs. Lwt for Threads

Show HN: If you lose your memory, how to regain access to your computer?

An Update on Heroku

Dark Alley Mathematics

PC Floppy Copy Protection: Vault Prolok

Was Benoit Mandelbrot a hedgehog or a fox?

Jeffrey Snover: "Welcome to the Room"

How to effectively write quality code with AI

Introducing the Developer Knowledge API and MCP Server

I spent 5 years in DevOps – Solutions engineering gave me what I was missing

Vocal Guide – belt sing without killing yourself

Female Asian Elephant Calf Born at the Smithsonian National Zoo

Why I Joined OpenAI

Understanding Neural Network, Visually

Show HN: R3forth, a ColorForth-inspired language with a tiny VM

I now assume that all ads on Apple news are scams

FORTH? Really!?

Zlob.h 100% POSIX and glibc compatible globbing lib that is faste and better

Learning from context is harder than we thought

Show HN: CallFS – S3-style object store in one Go binary (MIT)

Comments