frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

Open in hackernews

Tell HN: The Hetzner Experience - Invisible Outages

21•AmazingTurtle•8h ago
I'm a DevOps engineer at Schäfer Shop GmbH, and we've been running into recurring quirks with Hetzner's cloud infrastructure. Occasionally, volumes randomly disappear or become unavailable, with no indication on Hetzner's status page. Support ticket response times vary wildly - from hours to an entire business day.

Yesterday, we had a particularly stressful incident involving Hetzner load balancers in Falkenstein. Our Kubernetes control planes were unreachable due to load balancer targets showing as unhealthy. We quickly worked around the issue by deploying an identical load balancer configuration in another region. Despite explicitly instructing Hetzner support not to recreate our resources (since they're managed via Terraform), they manually recreated the load balancer anyway, causing momentary panic - though thankfully our Terraform state wasn't impacted.

We pay nearly €20,000 per month for Hetzner's services, yet they refuse to offer a direct support hotline, even if we were willing to pay extra for it. What's especially troubling is their persistent silence on these outages. Hetzner's status page showed no signs of this incident, neither during nor after. This pattern makes us question the transparency and purpose of the status page itself.

Have any of you experienced similar invisible outages with Hetzner?

Comments

herbst•7h ago
I know Hetzner just from a small customer perspective, I think I never had anything else than their discount servers.

However, I don't run anything mission critical with them as they don't really have a reliable support. Just using their cheap dedis for background tasks.

The server I have right now is stable, but I had different experiences before as well. And their network is unreliable either way, timeouts etc...

preisschild•7h ago
We have 100s of HCloud machines and never encountered similar issues, fortunately.

Just the typical server outages and a "Fault report" notification Email from Hetzner

jeduardo•6h ago
6 years ago when I used Hetzner, it was widely known as unreliable, a provider where you could get hosting that delivered good performance for a cheap price. The tradeoff always was that you needed to treat each machine almost as EC2 Spot: it could go down at any time.

You also needed to consider that when this happened, the data inside the machine was mostly lost. Finally, you also needed to plan to graduate out of it as soon as you had enough money to go either to a colocated data center or the "real cloud".

I kept Hetzner as a backup provider in more than one company, mainly to have real machines for take home tests, back when hiring was plentiful. Even so, we often faced problems with the machines going down due to hardware or networking issues, and the need to rebuild them from the ground up. Those mirrored all tales of woe everyone in the department had from years of working with Hetzner, sometimes losing production data because the rules of the game were not followed.

So it seems that 6 years later their scale has increased but the experience remains the same. On the bright side, kudos to Hetzner for teaching waves of engineers about reliability and disaster recovery during all these years.

jesterson•6h ago
> What's especially troubling is their persistent silence on these outages.

Based on many years of experience, all providers are guilty of that. Only large scale outages or ones that just couldnt be ignored are reflected on status page. This doesn't necessarily mean malevolence on provider side - their sensors just may not be good enough to spot the issue.

On larger scale - why would you choose hetzner and then complain about uptime? Its a well know provider with low prices and low reliability. There are tons of businesses who find this model suitable for them. If yours is not one of them - just switch to something more reliable. Granted, your bill will likely be 2x+ of 20k eur, but you get what you whine about.

As old adage says, we can make this project fast, cheap and with amazing quality, but you can choose only 2 options.

johcard•5h ago
We've had a very similar experience over the past year. We've been using Hetzner for over a decade and, until recently, we were really satisfied with their services. But in the last 12 months, the reliability has noticeably dropped.

Most of the issues we've faced are related to their Storage Boxes, multiple incidents where they were completely unavailable, sometimes for hours or even days. What’s frustrating is that these outages are never reflected on their status page, so you're left in the dark unless you open a ticket yourself. Even then, the only explanation we usually get is that the specific Storage Box is under "heavy load", and the suggested fix is always to migrate to another box. That might be fine for infrequent use, but it's not acceptable when you're relying on it.

To be fair, Hetzner has been a solid provider for many years, and we’re still hoping this is just a temporary rough patch. We really hope they get things back on track soon.

palata•1h ago
Could it be related to people in Europe suddenly looking for European alternatives? Like suddenly they got a lot of new customers in the last few months?
danielops•3h ago
Crazy to see this now as we also had a very similar issue yesterday! All backend nodes of our kubernetes cluster were suddenly and inexplicably showing as unhealthy despite being all green on the cluster side, no signs of issues whatsoever.
adamcharnock•3h ago
We’ve used Hetzner dedicated servers for many years now, but not cloud.

Our experience has been excellent, but we also design for the platform. Redundant dedicated networking, multi-AZ, networking failover, RAID, k8s, Mayastor, etc.

The worst issues we see are occasional scheduled outages of an upstream router. This will take out an AZ for external traffic for about 20 mins, but the internal dedicated network will ensure internal services all stay up and quorate.

It’s not cheap, but it’s still cheaper than AWS.

I think their dedicated offering is probably more stable as it has been around longer, and it is also much much simpler. They need to provide networking, power, and finance the hardware. All of which is very much solved problems.

(We’re https://lithus.eu, if anyone is interested. You can contact me at adam@…. I’m on holiday this week, but back next week)

oulipo•1h ago
I've had a lot of issues with resources located at Falkenstein, there seems to be issue at that particular location. I've moved to other locations, and so far it's running fine
rgavuliak•1h ago
In my first job we've used Hetzner for Data Science work. The we lost servers twice in a year back then.

Ask HN: What's behind the strong anti-AI sentiment on Hacker News?

6•cloudking•3h ago•23 comments

Tell HN: The Hetzner Experience - Invisible Outages

21•AmazingTurtle•8h ago•10 comments

Ask HN: Does the languages we speak affect the way we think?

2•Genius_um•4h ago•2 comments

Ask HN: Do people actually pay for small web tools?

48•scratchyone•2d ago•54 comments

Ask HN: When do you just give up and ship it?

12•90s_dev•13h ago•6 comments

How to Fix the Gaming Industry

2•azyLum•9h ago•4 comments

I'm Peter Roberts, immigration attorney, who does work for YC and startups. AMA

253•proberts•4d ago•451 comments

Ask HN: Don't You Mind That LLMs Are Mostly Proprietary?

9•dakiol•17h ago•12 comments

Big Beautiful Bill R&D Tax: Will tech go on a hiring spree again?

13•jbverschoor•1d ago•15 comments

Ask HN: How do you use AI for development in high security environments?

6•thesurlydev•19h ago•2 comments

Ask HN: Is there a Wikipedia or LLM wrapper for kids (with parental controls)?

4•soferio•14h ago•6 comments

Ask HN: What's your go-to message queue in 2025?

58•enether•5d ago•96 comments

Ask HN: We built a travel app – a classic tarpit idea. What now?

4•kenforthewin•22h ago•5 comments

Ask HN: Conversational AI to Learn a Language

14•edweis•3d ago•6 comments

Ask HN: How do you handle licensing and revenue leaks for self-hosted software?

3•lexokoh•22h ago•2 comments

Ask HN: Tech Behind the "Magic" Company?

4•ChicagoBoy11•22h ago•2 comments

Ask HN: When will managers be replaced by AI?

53•GianFabien•12h ago•71 comments

What interesting things low spending people do, that others know nothing about?

14•evolve2k•1d ago•34 comments

Ask HN: Anyone working in traditional ML/stats research instead of LLMs?

20•itsmekali321•3d ago•10 comments

Ask HN: Best on device LLM tooling for PDFs?

4•martinald•1d ago•1 comments

Ask HN: Moving to London from California

12•siamese_puff•1d ago•13 comments

Is current state of querying on observability data broken?

12•pranay01•2d ago•0 comments

Ask HN: Do you have a side project making more than $100 monthly?

12•leonagano•1d ago•4 comments

Ask HN: What newsletters do you follow?

5•cyndunlop•21h ago•4 comments

Ask HN: Email Provider for Main Account?

24•agent008t•5d ago•27 comments

Best AI editor for local models?

7•rocketbro•3d ago•0 comments

Ask HN: How do you store the knowledge gained in a day?

67•dennisy•6d ago•95 comments

Xray: A full-behavior-chain anti-malware system built in Go by a student

6•tangtian•3d ago•1 comments

Ask HN: How do you store private keys?

7•max_•3d ago•16 comments