frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

We cut our Mongo DB costs by 90% by moving to Hetzner

https://prosopo.io/blog/we-cut-our-mongodb-costs-by-90-percent/
70•arbol•1h ago

Comments

CyanLite2•1h ago
"We replaced a cluster of virtualized servers with a single bare metal server. Nothing has gone wrong, yet."
usrnm•1h ago
There are many cases when some downtime is perfectly ok. Or, at least, worth the savings
tayo42•1h ago
They saved a little under 3k and were motivated by the aws outage
arbol•1h ago
Lol yep that could've been the headline. We plan to add replica servers at some point. This DB is not critical to our product hence the relaxed interim setup.
rmoriz•1h ago
I‘m a big fan of owning the stack but why not spend the money on redundancy? At least a couple of machines in a different data center at Hetzner or another provider (OVH, Scaleway, Vultr, …) can easily fit your budget.
arbol•1h ago
We will be adding additional db servers and running our own replica set eventually. We're just not there yet. Thanks for reading!
petcat•1h ago
> The more keen eyed among you will have noticed the huge cost associated with data transfer over the internet - its as much as the servers! We're building Prosopo to be resilient to outages, such as the recent massive AWS outage, so we use many different cloud providers.

I mean, you're connecting to your primary database potentially on another continent? I imagine your costs will be high, but even worse, your performance will be abysmal.

> When you migrate to a self-hosted solution, you're taking on more responsibility for managing your database. You need to make sure it is secure, backed up, monitored, and can be recreated in case of failure or the need for extra servers arises.

> ...for a small amount of pain you can save a lot of money!

I wouldn't call any of that "a small amount of pain." To save $3,000/month you've now required yourself to become experts in a domain that maybe is out of your depth. So whatever cost saved is now tech debt and potentially having to hire someone else to manage your homemade solution for you.

However, I self-host, and applaud other self-hosters. But sometimes it really has to make business sense for your team.

arbol•59m ago
> I mean, you're connecting to your primary database potentially on another continent?

Atlas AWS was actually setup in Ireland. The data transfer costs were coming from extracting data for ML modelling. We don't get charged for extracting data with the new contract.

> experts in a domain that maybe is out of your depth

We're in the bot detection space so we need to be able to run our own infra in order to inspect connections for patterns of abuse. We've built up a fair amount of knowledge because of this and we're lucky enough to have a guy in our team who just understands everything related to computers. He's also pretty good at disseminating information.

Thanks for reading!

goastler•36m ago
aww shucks ;)
kachapopopow•1h ago
Always consider if 12 hours of lost revenue is worth the savings. Recently hetzner has been flakey with minimum or no response for support or even status updates that anything was wrong. My favorite was them blaming an issue on my side just to have a maintenance status update the day after about congestion.
arbol•1h ago
Atlas wasn't giving us any support for $3K per month. Hetzner at least have some channel to contact them, which is an improvement. That said, if their uptime is rubbish them we'll probably migrate again. Moving back to Atlas is not an option as we were getting hammered by the data transfer costs and this was only going to increase due to our architecture. Thanks for reading!
zamalek•55m ago
OVH is allegedly pretty good. I host all my personal stuff on Hetzner right now so I can't speak to it personally.
arbol•49m ago
We also use OVH and have so far not had any downtime in about 6 months.
kosherhurricane•49m ago
500GB isn't a lot of data, and $3K/month seems like an extortion for that little data.

Having said that, MongoDB pricing page promises 99.995% uptime, which is outstanding, and would probably be hard to beat that doing it oneself, even after adding redundancy. But maybe you don't need that much uptime for your particular use case.

arbol•36m ago
Its more like 700GB now on the new server and we were about to have to migrate to a higher tier on Atlas.

> maybe you don't need that much uptime for your particular use case.

Correct. Thanks for reading!

0x073•58m ago
Using hetzner since 5 years never had issues and only 1 downtime in one data center.
izacus•6m ago
My Hetzner instances all have higher reliability and uptime than AWS deployments. For years now.

That was an interesting surprise.

CodesInChaos•1h ago
MongoDB Atlas is so overpriced that you can probably save already 90% by moving to AWS.
computerfan494•1h ago
Most of the cost in their bill wasn't from MongoDB, it was cost passed on from AWS
CodesInChaos•46m ago
I don't remember the numbers (90% is probably a bit exaggerated) but our savings of going from Atlas to MongoDB Community on EC2 several years ago were big.

In addition to direct costs, Atlas had also expensive limitations. For example we often spin up clone databases from a snapshot which have lower performance and no durability requirements, so a smaller non-replicated server suffices, but Atlas required those to be sized like the replicated high performance production cluster.

cnkk•1h ago
Are you sure you went with RAID1 with 4x disks instead of RAID10?
arbol•49m ago
Good spot - this is wrong. It should've been 4 x 3.84 TB NVMe SSD RAID 5. My colleague set this bit up so I'm not entirely up to speed on the terminology.
CodesInChaos•59m ago
How long does mongodump take on that database? My experience was that incremental filesystem/blockdevice snapshots were the only realistic way of backing up (non sharded) mongodb. In our case EBS snapshots, but I think you can achieve the same using LVM or filesystems like XFS and ZFS.
goastler•38m ago
It takes ~21hrs to dump the entire db (~500gb), but I'm limited by my internet speed (100mbps, seeing 50-100mbps during dump). Interestingly, the throughput is faster than doing a db dump from atlas which used to max around 30mbps
0xbadcafebee•56m ago
> Here's how we managed to cut our costs by 90%

You could cut your MongoDB costs by 100% by not using it ;)

> without sacrificing performance or reliability.

You're using a single server in a single datacenter. MongoDB Atlas is deployed to VMs on 2-3 AZs. You don't have close to the same reliability. (I'm also curious why their M40 instance costs $1000, when the Pricing Calculator (https://www.mongodb.com/pricing) says M40 is $760/month? Was it the extra storage?)

> We're building Prosopo to be resilient to outages, such as the recent massive AWS outage, so we use many different cloud providers

This means you're going to have multiple outages, AND incur more cross-internet costs. How does going to Hetzner make you more resilient to outages? You have one server in one datacenter. Intelligent, robust design at one provider (like AWS) is way more resilient, and intra-zone transfer is cheaper than going out to the cloud ($0.02/GB vs $0.08/GB). You do not have a centralized or single point of failure design with AWS. They're not dummies; plenty of their services are operated independently per region. But they do expect you to use their infrastructure intelligently to avoid creating a single point of failure. (For example, during the AWS outage, my company was in us-east-1, and we never had any issues, because we didn't depend on calling AWS APIs to continue operating. Things already running continue to run.)

I get it; these "we cut bare costs by moving away from the cloud" posts are catnip for HN. But they usually don't make sense. There's only a few circumstances where you really have to transfer out a lot of traffic, or need very large storage, where cloud pricing is just too much of a premium. The whole point of using the cloud is to use it as a competitive advantage. Giving yourself an extra role (sysadmin) in addition to your day job (developer, data scientist, etc) and more maintenance tasks (installing, upgrading, patching, troubleshooting, getting on-call, etc) with lower reliability and fewer services, isn't an advantage.

goastler•25m ago
> you're going to have multiple outages us: 0, aws: 1. Looking good so far ;)

> AND incur more cross-internet costs hetzner have no bandwidth traffic limit (only speed) on the machine, we can go nuts.

I understand you point wrt the cloud, but I spend as much time debugging/building a cloud deployment (atlas :eyes: ) as I do a self-hosted solution. Aws gives you all the tools to build a super reliable data store, but many people just chuck something on us-east-1 and go. There's you single point of failure.

Given we're constructing a many-node decentralised system, self-hosted actually makes more sense for us because we've already had to become familiar enough to create a many-node system for our primary product.

When/if we have a situation where we need high data availability I would strongly consider the cloud, but in the situations where you can deal with a bit of downtime you're massively saving over cloud offerings.

We'll post a 6-month and 1-year follow-up to update the scoreboard above

euph0ria•52m ago
You probably want to store the backup somewhere else, ie. not Hetzner.

They are known to just cancel accounts and cut access.

sdoering•49m ago
Any proof of that? I am a Hetzner customer and had never heard of this before. Would be good to know what I got into.
ch2026•37m ago
A few years back I launched an io game and used hetzner as my backend. an hour into launch day they null routed my account because their anti-abuse system thought my sudden surge in websocket connections was an attack (unclear if they thought it was inbound or outbound doing the attacking).

I had paid for advertising on a few game curation sites plus youtubers and streamers. Lovely failure all thanks to Hetzner. Took 3 days and numerous emails with the most arrogant Germans you’ve ever met before my account was unlocked.

I switched to OVH and while they’re not without their own faults (reliability is a big one), it’s been a far better experience.

__turbobrew__•17m ago
OVH also null routes, it has happened to me.

It seems like you have to go to one of the big boys like hurricane electric where you are allowed to use the bandwidth you paid for without someone sticking their fingers in it.

arcanemachiner•35m ago
There are a lot of such stories if you go digging around HN and reddit threads. Haven't seen a lot of these stories in a while, so it may be happening less now.
arbol•43m ago
Good shout. I think we'll also run replicas on other providers. We've got some complex geo-fencing stuff to do with regards to data hence why we're just on Hetzner right now.
PeterZaitsev•49m ago
Note, if you're looking for MongoDB Enterprise features you can find many of them with Percona Server for MongoDB, which you can use for free the same way as MongoDB Community
arbol•45m ago
Nice, thanks for the tip!
ianberdin•45m ago
I’m starting to worry about this Hetzner trend. It can end up to get the price skyrocketing.
arbol•42m ago
Hopefully not. Their console is pretty bad so I reckon that will put a lot of people off.
patrickmcnamara•40m ago
The new console is completely fine.
the_duke•39m ago
The cloud console is pretty good though? Even does live sync!

The old one for dedicated servers (robot) is horribly outdated though.

arbol•35m ago
Ah right, we're on robot so I've not seen the cloud one. Robot is old! :)
goastler•39m ago
There's other providers (OVH, etc) so I'm sure the price will remain competitive
righthand•38m ago
We’re just going to end up with everyone moving from Amazon to Hetzner and the same issue will remain. High prices, lockin, etc will appear.

We need an American “get off American big tech” movement.

Differentiate people! Reading “we moved from X to Y” does not mean everyone move from X to Y, it means start considering the Y values and research other Y’s around you.

arbol•34m ago
We also use OVH, Contabo, Hostwinds... Architect so you can be multi-provider and reduce internet centralisation!
righthand•21m ago
Nice, if you write an article about it, try to leave the focus off of a single hosting provider. Encouraging the differentiation is important too (next time! I’m not dogging the movement or your efforts in this article, I love to see reduced reliance of Amazon in general).
dehrmann•1m ago
EC2 is sort of a ceiling price.
poszlem•35m ago
As in so many of these stories, what gets glossed over is just how much complexity there is in setting up your own server securely.

You set up your server. Harden it. Follow all the best practices for your firewall with ufw. Then you run a Docker container. Accidentally, or simply because you don’t know any better, you bind it to 0.0.0.0 by doing 5432:5432. Oops. Docker just walked right past your firewall rules, ignored ufw, and now port 5432 is exposed with default Postgres credentials. Congratulations. Say hello to Kinsing.

And this is just one of many possible scenarios like that. I’m not trying to spread FUD, but this really needs to be stressed much more clearly.

EDIT. as always - thank you HN for downvoting instead of actually addressing the argument.

mkesper•9m ago
I don't see the point of using ufw at all as Hetzner provides an external firewall.
poszlem•7m ago
If you use a dedicated hetzner machine you only get a stateless firewall. That would be one reason.
cpursley•28m ago
Why in the world do people choose Mongo over Postgres? I'm legit curious. Is it inexperience? Javascript developers who don't know backend or proper data modeling (or about jsonb)? Is this type of decision coming down from non-technical management? Are VCs telling their portfolio companies what to use so they have something to burn their funding on? It's just really confounding, especially when there's even mongo-api compatible Postgres solutions now. Perhaps I'm just not webscale and too cranky.
nalekberov•17m ago
IMHO it's because so many people take decisions in rush. e.g. let's not design database, put whatever data shape we came ip in alpha version and see where it goes. Sometimes people favor one particular technology because every other startup chose it.

To be quite honest today's software engineering sadly is mostly about addressing 'how complex can we go' rather than 'what problem are we trying to solve'.

a13n•15m ago
maybe instead of communicating how dumb you think people are for choosing mongo, communicate why you think it’s so dumb
zkmon•16m ago
Atlas is plain robbery. I see companies paying 600K USD/month on a few clusters, mostly used for testing. The problem is they got locked into this, by doing a huge migration of their apps and switching to a different tech would easily take 2 to 5 years.

Britain's railway privatization was an abject failure

https://www.rosalux.de/en/news/id/53917/britains-railway-privatization-was-an-abject-failure
349•robtherobber•3h ago•275 comments

Zed Is Our Office

https://zed.dev/blog/zed-is-our-office
76•sagacity•1h ago•12 comments

GitHub Partial Outage

https://www.githubstatus.com/incidents/1jw8ltnr1qrj
75•danfritz•1h ago•36 comments

Launch HN: Tweeks (YC W25) – Browser extension to de-enshittify the web

https://www.tweeks.io/onboarding
24•jmadeano•51m ago•21 comments

Hemp Ban Hidden Inside Government Shutdown Bill

https://hightimes.com/news/politics/hemp-ban-hidden-inside-government-shutdown-bill/
45•bilsbie•1h ago•7 comments

Checkout.com hacked, refuses ransom payment, donates to security labs

https://www.checkout.com/blog/protecting-our-merchants-standing-up-to-extortion
369•StrangeSound•7h ago•193 comments

Blender Lab

https://www.blender.org/news/introducing-blender-lab/
101•radeeyate•3h ago•32 comments

Android developer verification: Early access starts

https://android-developers.googleblog.com/2025/11/android-developer-verification-early.html
1220•erohead•16h ago•552 comments

Kratos - Cloud native Auth0 open-source alternative (self-hosted)

https://github.com/ory/kratos
55•curtistyr•2h ago•33 comments

SIMA 2: An Agent That Plays, Reasons, and Learns with You in Virtual 3D Worlds

https://deepmind.google/blog/sima-2-an-agent-that-plays-reasons-and-learns-with-you-in-virtual-3d...
33•meetpateltech•1h ago•5 comments

European Nations Decide Against Acquiring Boeing E-7 Awacs Aircraft

https://defensemirror.com/news/40527/European_Nations_Decide_Against_Acquiring_Boeing_E_7_AWACS_A...
76•saubeidl•1h ago•67 comments

Denx (a.k.a. U-Boot) Retires

https://www.denx.de/
44•synergy20•2h ago•9 comments

The Monks in the Casino

https://www.derekthompson.org/p/the-monks-in-the-casino
8•pavel_lishin•34m ago•2 comments

We cut our Mongo DB costs by 90% by moving to Hetzner

https://prosopo.io/blog/we-cut-our-mongodb-costs-by-90-percent/
71•arbol•1h ago•51 comments

COBOL to Kotlin via Formal Models (IR and Alloy and Golden Master)

https://marcoeg.medium.com/from-cobol-to-kotlin-795920b1f371
16•marcoeg•5d ago•1 comments

Tesla Is Recalling Cybertrucks Again. Yep, More Pieces Are Falling Off

https://www.popularmechanics.com/cars/hybrid-electric/a69384091/cybertruck-lightbar-recall/
112•2OEH8eoCRo0•1h ago•48 comments

Heartbeats in Distributed Systems

https://arpitbhayani.me/blogs/heartbeats-in-distributed-systems/
31•sebg•3h ago•6 comments

Switching from GPG to Age

https://luke.hsiao.dev/blog/gpg-to-age/
66•speckx•1w ago•38 comments

Human Fovea Detector

https://www.shadertoy.com/view/4dsXzM
361•AbuAssar•16h ago•77 comments

Android 16 QPR1 is being pushed to the Android Open Source Project

https://grapheneos.social/@GrapheneOS/115533432439509433
204•uneven9434•13h ago•99 comments

Telli (Voice AI – YC F24) is hiring engineers in Berlin

https://hi.telli.com/eng
1•sebselassie•7h ago

A Challenge to Roboticists: My Humanoid Olympics

https://spectrum.ieee.org/humanoid-robot-olympics
19•quapster•1w ago•4 comments

Steam Machine

https://store.steampowered.com/sale/steammachine
2457•davikr•22h ago•1149 comments

Seed. LINE's Custom Typeface

https://seed.line.me/index_en.html
77•totetsu•7h ago•36 comments

Reverse Engineering Yaesu FT-70D Firmware Encryption

https://landaire.net/reversing-yaesu-firmware-encryption/
98•austinallegro•9h ago•14 comments

Homebrew no longer allows bypassing Gatekeeper for unsigned/unnotarized software

https://github.com/Homebrew/brew/issues/20755
299•firexcy•19h ago•230 comments

Shader Glass

https://github.com/mausimus/ShaderGlass
56•erickhill•5d ago•10 comments

GPT-5.1: A smarter, more conversational ChatGPT

https://openai.com/index/gpt-5-1/
480•tedsanders•21h ago•605 comments

Continuous Autoregressive Language Models

https://arxiv.org/abs/2510.27688
87•Anon84•1w ago•6 comments

Transpiler, a Meaningless Word (2023)

https://people.csail.mit.edu/rachit/post/transpiler/
98•jumploops•6d ago•80 comments