Cloudflare Global Network experiencing issues

https://www.cloudflarestatus.com/?t=1

589•imdsm•1h ago

Comments

imdsm•1h ago

Related to Azure DDoS?

https://news.ycombinator.com/item?id=45955900

alexdns•1h ago

whats your chain of thought here ? a company that has nothing to do with azure is down because azure got ddosed 2 weeks ago ?

Jordan-117•1h ago

Maybe that any actor sophisticated enough to take down Azure might also target Cloudflare?

hirako2000•33m ago

Especially when the next happens to be a major DDOS mitigator.

polisaez•1h ago

Maybe related to their scheduled maintenance? https://www.cloudflarestatus.com/

chedabob•1h ago

I thought that as we're seeing issues with LON, but their Manchester POP is also down and that didn't have any maintenance this morning.

pstation•1h ago

Ironically I can't even read the link in the article because cloudflare is down.

watermelon0•48m ago

Linked Microsoft blog article mentions that DDoS was in October.

its_notjack•1h ago

Ironically, DownDetector seems to be down because it protects its site with Cloudflare Turnstile... which is also down!

imdsm•1h ago

I noticed this too!

thrdbndndn•1h ago

The report there for AWS also skyrocketed, but I guess it's probably false positives?

Macha•48m ago

Even many non tech people have begun to associate Internet wide outages with “aws must be down” so I imagine many of them searching “is aws down” and for down detector, a hit is a down report, so it will report aws impacts even when the culprit is cloudflare in this case

mailund•45m ago

interesting, maybe "AWS is down" will become the new "the server is down" that some non-tech people throw around when anything unexpected happen on their computer?

itzjacki•1h ago

A colleague of mine just came bursting through my office door in a panic, thinking he brought our site down since this happened just as he made some changes to our Cloudflare config. He was pretty relieved to see this post.

Freak_NL•1h ago

Is there a word for that feeling of relief when someone else fucked up after initially thinking it was you?

StanAngeloff•1h ago

Schadenfreude

simonklitj•1h ago

Not quite, that’s more like taking pleasure in the misfortune of someone else. It’s close, but the specific relief bit that it is not _your_ misfortune is not captured

skottenborg•47m ago

Internettet er vist ikke så stort :-)

gnfargbl•1h ago

Nah, that's delight in someone else's misfortune. This is delight that the misfortune wasn't yours, which is slightly different.

StanAngeloff•1h ago

4 years of German and I still don't quite "get" it :^) TY!

tagyro•55m ago

We have a saying:

You know how you measure eternity?

When you finish learning German.

jpmonette•1h ago

phewphoria

Freak_NL•1h ago

Yup, that works.

Sholmesy•48m ago

Well, at least something good came out of this incident.

Perfect.

zzzeek•4m ago

it has to sound like a german word though

shortrounddev2•52m ago

When I'm debugging something, I'm not usually looking for the solution to the problem; I'm looking for sufficient evidence that I didn't cause the problem. Once I have that, the velocity at which I work slows down

cromka•52m ago

vindication?

jspash•46m ago

The problem is, I still get the wrong end of the stick when AWS or CF go down! Management doesn't care, understandably. They just want the money to keep coming in. It's hard to convince them that this is a pretty big problem. The only thing that will calm them down a bit is to tell them Twitter is also down. If that doesn't get them, I say ChatGPT is also down. Now NOBODY will get any work done! lol.

graemep•15m ago

Who decided to go with AWS of CF? If its a management decision tell them you need the resources to have a fallback if they want their system to be more reliable than AWS or CF.

mcphage•3m ago

Maybe this isn’t great, but I get a hint of that feeling when I’m on an airplane and hear a baby crying. For a number of years, if I heard a baby crying, it was probably my baby and I had to deal with it. But now my kids are past that phase, so when I hear the crying, after that initial jolt of panic I realize that it isn’t my problem, and that does give me the warm fuzzies. Even though I do feel bad for the baby and their parents.

NitpickLawyer•55m ago

You missed a great opportunity to dead-pan him with something like "No, Bob, not just our site, you brought down the entire Internet, look at this post!"

sakisv•53m ago

Well, you can never be sure that he didn't:

https://www.fastly.com/blog/summary-of-june-8-outage

itzjacki•51m ago

Oh don't you worry. We are very much talking about the global outage as if he was the root cause. Like good colleagues :)

rapnie•10m ago

Hmm, wait a minute.. maybe he was the cause! (no, kidding. just upping the pressure as a good peer :)

belter•48m ago

Wait for the post mortem ... It is a technical possibility, race condition propagates one customer config to all nodes... :-)

bamboozled•47m ago

How do we know your colleagues changes didn't take down Cloudflare though?

itzjacki•44m ago

Good point. We should probably assume they did, until proven otherwise.

disconnection•22m ago

Good that he worked it out so quick. I recently spent a day debugging email problems on Railway PaaS, because they silently closed an SMTP port without telling anyone.

nrhrjrjrjtntbt•19m ago

The company where this colleague works? Cloudflare.

vldszn•1h ago

Also seeing this on my websites hosted on cloudflare

Folyd•1h ago

Insane, my website https://geddle.com totally down

dutchbrit•1h ago

seems to work here

Folyd•1h ago

down again, all my website rely on Cloudflare DNS are down

robotfelix•1h ago

Back up for me now

fspoettel•1h ago

Down again

robotfelix•1h ago

Yep - down again for me too!

ilaksh•1h ago

When this kind of thing happens it makes me feel better about my own programming problems.

I wonder if it has anything to do with the replicate.com purchase? Probably not.

bratao•1h ago

The danger of Internet centralization in Cloudflare

snitty•41m ago

That's why I run my server on 7100 chips made for me by Sam Zeloof in his garage on a software stack hand coded by me, on copper I ran personally to everyone's house.

r721•1h ago

Another thread: https://news.ycombinator.com/item?id=45963781

UPD And https://news.ycombinator.com/item?id=45963949

sammy2255•1h ago

Turnstile is throwing 500 internal server error

js4ever•1h ago

Cloudflare fully down for me and my team, half of internet just vanished

teitoklien•1h ago

Same we use cloudflare as image cdn and its R2 service a lot, cant access anything over its cdn route

re-thc•1h ago

Downflare

DC-3•1h ago

Cloudflare'nt

Terr_•1h ago

I'm thinking about all those quips from a few decades back, along the lines of: "The Internet is resilient, it's distributed and it routes around damage" etc.

In many ways it's still true, but it doesn't feel like a given anymore.

StanAngeloff•1h ago

No logging in to Cloudflare Dash, no passing Turnstile (their CAPTCHA Replacement Solution) on third-party websites not proxied by Cloudflare, the rest that are proxied throwing 500 Internal server error saying it's Cloudflare's fault…

Feels like half the internet is down.

woodylondon•1h ago

my cloudflare pages website is down - 500 server error :(

cannot login to get to workers to check - auth errors

I thought this was the point of a cached CDN!

Mojah•1h ago

Interestingly, also noticing that websites that use Cloudflare Challenge (aka "I'm not a Robot") are also throwing exceptions with a message as "Please unblock challenges.cloudflare.com to proceed" - even though it's just responding with an HTTP/500.

gwd•1h ago

I do appreciate the visual "mea culpa":

Your browser: Working

Host: Working

Cloudflare: Error

mallets•1h ago

Might be the first time I have ever seen that. Though in my case the "Host" is Cloudflare's own Pages service.

udev4096•50m ago

Yeah, I was shocked. Disbelief that the host was up, which is what usually happens when the cloudflare's page show up

janmo•1h ago

And at the bottom:

What can I do?

Please try again in a few minutes.

mig4ng•59m ago

That is really good to be honest!

I have Cloudflare running in production and it is affecting us right now. But at least I know what is going on and how I can mitigate (e.g. disable Cloudflare as a proxy if it keeps affecting our services at skeeled).

Lukas_Skywalker•56m ago

They still blame the customers when you click on "Cloudflare":

> If the problem isn’t resolved in the next few minutes, it’s most likely an issue with the web server you were trying to reach.

mig4ng•52m ago

In terms of probability looking at the history, it is correct. It's mostly me messing up with the web server.

jap•51m ago

That page has special if/endif HTML comments to handle if your browser is IE 6, IE 7, IE 8...

WhereIsTheTruth•1h ago

West's Great Firewall

alexcroox•1h ago

It's been 15 minutes of it going up and down, still nothing on their status page...

imdsm•1h ago

Well, we've seen it now, they'll have to update it eventually!

timpera•1h ago

They've just added it less than a minute ago. I expected a little more responsiveness from Cloudflare...

arkx•1h ago

Me too. What good is a status page that's not automated?

timpera•53m ago

Especially when it's the de facto status page for 20% of the internet!

ArcHound•52m ago

I can answer that - once the lawyers take interest in your SLAs, you need to check with them if this is really an incident. Otherwise, you might lose some contract money and nobody wants that.

rco8786•4m ago

There are no truly automated status pages. It's an impossible problem.

rco8786•4m ago

Everything else aside, 20 minutes to get their status page updated seems pretty damn fast.

karimf•1h ago

I've been migrating all my personal stuff to Cloudflare. They have good products and good pricing.

At the same time I'm worried about how the internet is becoming even more centralized, which goes against how it was originally designed.

immibis•1h ago

Cognitive dissonance

minitech•55m ago

No, just competing priorities.

kilroy123•53m ago

Same here. A lot of my sites are now down.

fny•1h ago

Ironically, cloudflare.com is not down.

ed_mercer•51m ago

Yeah I wonder how that works

Dilettante_•6m ago

"Don't get high on your own supply"?

davisonio•1h ago

Even downdetector is down, I can't get through the Cloudflate captcha.

rvz•1h ago

Working just fine in the UAE.

tokioyoyo•1h ago

I had two completely unrelated tabs open (https://twitter.com and https://onsensensei.com) both showing the same error. Opened another website, same error. Kinda funny to see how much of entire web is ran on CloudFlare nowadays.

burtekd•1h ago

Love how everyone plays with redundancy - multiple hosts, balance loader, etc, and yet half of the web relies on single point of failure being CF

6031769•43m ago

Indeed. And it feels really good knowing that our stuff isn't in that half.

lofaszvanitt•1h ago

So it begins. Now is the time to banish the evil presence from the internet. :D

Ekaros•1h ago

I was joking that after AWS, Azure that the Cloudflare would be next one...

So which large service we have left that could take chunk of internet out?

sammy2255•1h ago

Gcloud

entropoem•1h ago

Everyone laughs when AWS collapses, everyone is silent when Cloudflare collapses. Why? Because the place to laugh has collapsed.

Hedepig•1h ago

Except on HN

HAHA!

Our servers are still down, though

sutton12345•56m ago

HAHAHAHA

imdsm•52m ago

I'm not sure whether to laugh or cry

Maybe I'll do both

sutton12345•39m ago

YEP that's the case nowwwww

belter•52m ago

One is every seven years... the other one is a ...monthly event?: https://hn.algolia.com/?https://hn.algolia.com/?dateRange=al...

ceejayoz•47m ago

Most of those aren’t outages, and both providers have big blips.

Globally meaningful outages of either are quite rare.

mvdtnz•37m ago

This is at least the third time in the past year that Cloudflare has caused a significant outage of my product.

6031769•34m ago

And you are still using them because ... ?

mvdtnz•23m ago

We handle ~2M requests per second and CF eliminates about ⅔ of those. We need CF or something like it. Multi edge is harder than it sounds at very large scale.

viraptor•15m ago

There are still alternatives like Bunny https://status.bunny.net/history (may not be for everyone, but I like to post the CF alternatives so it becomes ever so slightly less of a default)

belter•36m ago

Its actually way more than monthly... It seems nobody does outages like Cloudflare...

https://www.cloudflarestatus.com/history?page=8

https://www.cloudflarestatus.com/history?page=7

https://www.cloudflarestatus.com/history?page=6

https://www.cloudflarestatus.com/history?page=5

https://www.cloudflarestatus.com/history?page=4

https://www.cloudflarestatus.com/history?page=3

https://www.cloudflarestatus.com/history?page=2

https://www.cloudflarestatus.com/history?page=1

curious_curios•30m ago

I at least respect them for reporting them. It feels like lots of cloud providers don’t, or begrudgingly.

AnkerSkallebank•51m ago

Everyone is silent when Cloudflare collapses. Same goes for Azure, but that is because noone uses it.

imdsm•15m ago

When Azure goes down: Oh well

When Cloudflare goes down: Oh no

exe34•50m ago

Down detector broke.... :-D

techguy1954•47m ago

Yeah, how ironic. The site that is designed to tell you if something else is down, is currently down.

dspillett•37m ago

Life imitates art, Red Dwarf ~1989:

    Lister: What's the damage Hol? 
    Holly:  I don't know. The damage report machine has been damaged.

tfiskgul•16m ago

It also reminds me of this part of Hitchhikers Guide to the Galaxy

https://www.penguinrandomhouse.ca/books/661/mostly-harmless-...

pelagicAustral•50m ago

OMG! today of all days!

nrhrjrjrjtntbt•21m ago

Black Tuesday

postalcoder•49m ago

I have been spam refreshing my cloudflare dns settings page and have managed to free https://hcker.news of its proxy. I feel like indiana jones and I just slid under a falling door.

tzpbingo•39m ago

Because those who were mocking him couldn't speak, X also crashed.

timeon•34m ago

This would be true in the past but now most people are not on Twitter.

JdeBP•21m ago

Sadly, I can report that this has brought down 2 of the major Mastodon nodes in the United Kingdom.

Happily, the small ones that I also use are still going without anyone apparently even noticing. At least, the subject has yet to reach their local timelines at the time that I write this.

2 of the other major U.K. nodes are still up, too.

r0fl•16m ago

Peter Levels wisdom about why to host not on aws not looking so wise right now

basilgohar•6m ago

There are places other than AWS to host.

matt-p•14m ago

Cloudflare down because of a DDOS is extremely funny.

sammy2255•13m ago

There's no evidence to suggest it was a result of a DDoS attack

simondotau•5m ago

That's true, but there is evidence that it would be extremely funny.

yen223•1h ago

It's down here in Sydney as well. The status page hasn't been updated to reflect that

helloguillecl•1h ago

Funny that I could not load Twitter to see if Cloudflare was down.

I rushed to Hacker News, but it was too early. Clicking on “new” did the job to find this post before making it to the Homepage:)

The web is still alive!

timeon•10m ago

It was on Mastodon. That one is hardly going down.

ksec•1h ago

HN has become the place to check if any HyperScaler + Cloudflare is down.

montekristooGDB•59m ago

while my colleagues are wondering why cloudlfare isn't working and are afraid it might be something from us locally, I'll first check here to make sure it's not a Cloudflare / AWS problem in the first place.

timpera•55m ago

When was HN down for the last time? :)

zwog•54m ago

I actually came here to check because downforeveryoneorjustme.com and downdetector are offline as well.

cipz•1h ago

Downflare

mchinen•1h ago

Down, but the linked status page shows mostly operational, except for "Support Portal Availability Issues" and planned maintenance. Since it was linked, I'm curious if others see differently.

edit: It now says "Cloudflare Global Network experiencing issues" but it took a while.

erdaltoprak•1h ago

At some point we really need to think if this is the web we want, one/two major actors are down and everything goes with it

Not downplaying the immense work of infra / engineering at this scale but my neighborhood local grocery market shouldn’t be down

eptcyka•1h ago

Decentralisation is at some point directly opposed to operational efficiency, when the sun is shining.

starbugs•34m ago

And centralization is ineffective long term

raxxorraxor•26m ago

A shark is an extremely energy efficient creature, but it is relatively stupid.

Dilettante_•17m ago

At some point every website will be the equivalent of Surströmming

lofaszvanitt•54m ago

Why everyone needs to be behind Cloudflare. I don't think DDOSing sites out of whim is so rampant that everyone needs the virtual umbrella.

frameset•52m ago

It actually is.

I run a small video game forum with posts going back to 2008. We got absolutely smashed by bots scraping for training data for LLMs.

So I put it behind Cloudflare and now it's down. Ho hum.

shaky-carrousel•47m ago

It'd funny if these bots were run by Cloudflare.

frameset•25m ago

Ha, yeah. They seemed to mostly be in SE Asia.

stevepotter•16m ago

Can you please elaborate on “smashed”? I’m very interested

watermelon0•9m ago

Have you tried Anubis or similar tools? I've had similar issues with bot scraping of a forum taking all server resources, and using PoW challenge solved the problem.

https://github.com/TecharoHQ/anubis

reassess_blind•51m ago

I’ve been DDoS’d countless times running a small scale, uncontroversial SaaS. Without them I would’ve had countless downtime periods with really no other way to mitigate.

dukeyukey•51m ago

Good chance the reason DDOSing isn't so big anymore is because everyone is on Cloudflare.

xslvrxslwt•25m ago

No but because all of us were arrested in 2018 for running DDoS-4-hire services. Bypassing cloudflare is very easy and I still can fry any of your websites (if i wanted to, just like any other skid)

grey413•51m ago

It's the web-scrapers. I run a tiny little mom and pop website, and the bots were consistently using up all of my servers' resources. Cloudfare more or less instantly resolved it.

jwr•15m ago

You mean you outsourced to Cloudflare the decision on who is allowed to view your website. That could be well-intentioned, but it's a risky thing to do, and I would not to outsource that decision. Especially as I wouldn't know who failed to get to my website as there is no way to appeal the decision.

As a side note, what does your site do that it's possible to use up all server resources? Computers are stupid fast these days. I find it's really difficult to build something that doesn't scale to at least multiple hundreds of requests per second.

luckylion•50m ago

It's not super common, but common enough that I don't want to deal with it.

The other part is just how convenient it is with CF. Easy to configure, plenty of power and cheap compared to the other big ones. If they made their dashboard and permission-system better (no easy way to tell what a token can do last I checked), I'd be even more of a fan.

If Germany's Telekom was forced to peer on DE-CIX, I'd always use CF. Since they aren't and CF doesn't pay for peering, it's a hard choice for Germany but an easy one everywhere else.

BoldColdHold•48m ago

DDOSing is absolutely so rampant that you need to be behind something.

lofaszvanitt•46m ago

Hm, interesting times we live in.

shaky-carrousel•46m ago

Nope, I'm at hetzner and haven't seen a DDoS in years.

xslvrxslwt•24m ago

Because of 2018 operation "Power OFF" but it's still pretty easy to take anything down.

Hetzner has the WEAKEST DDoS protection out of ANYTHING out there - Arbor sucks.

Send me your website url and I'll keep it down for DAYS and whenever you cry to hetzner I'll just fry it again, it's that easy and that's why they're the cheapest - because everyone ran away from them back then.

Mordisquitos•36m ago

Analogously, arson attacks against businesses in Palermo are absolutely so rampant that they need to be protected by someone.

timpera•33m ago

I run a few websites with moderate traffic (~900K daily page views total) on the same VPS and never had an issue with DDOS. Is this specific to some industries?

xslvrxslwt•18m ago

Literally specific to "did I make this skid angry or not", it takes $5 to DDoS a website (bypassing cloudflare included)

BoldColdHold•8m ago

Depends on what those websites are and how lucky you are.

chromehearts•48m ago

Cloudflare DDOS protection is super essential (especially for smaller businesses)

Tanath•42m ago

DDoS prevention may be essential, but not CloudFlare.

Mordisquitos•40m ago

Who is motivated to launch DDoS against smaller businesses? What do they have to gain?

xslvrxslwt•23m ago

Anyone that has $5.

Mordisquitos•14m ago

I've also got €5, but I see greater return on investment in spending them on a lottery ticket than in DDoS'ing arbitrary small businesses.

the_bear•8m ago

My small SaaS app has been DDoSed a handful of times, always accompanied by an email asking for a ransom in the form of bitcoin.

The first time we switched to Cloudflare which saved us. Even with Cloudflare, the DDoS attempts are still damaging (the site goes down, we use Cloudflare to block the endpoints they're targeting, they change endpoints, etc.) but manageable. Without Cloudflare or something like it, I think it's possible that we'd be out of business.

hennell•45m ago

Honestly it kinda is. Ai bots scrape everything now, social media means you can go viral suddenly, or you make a post that angers someone and they launch an attack just because. I default to cloudflare, because like an umbrella I might just be carrying it around most of the time, but in the case of a sudden downpoor it's better than getting wet.

isodev•43m ago

There are plenty of alternatives to protect against DDoSing, people like convenience though. “Nobody gets fired for choosing Microsoft/Cloudflare”. We have a culture problem

xslvrxslwt•26m ago

I was arrested by Interpol in 2018 because of warrants issued by the NCA, DOJ, FBI, J-CAT, and several other agencies, all due to my involvement in running a DDoS-for-hire website. Honestly, anyone can bypass Cloudflare, and anyone that want to take your website down - will take it down. It's just that luckily for all of us most of the DDoS-4-hire websites are down nowadays but there are still many botnets out there that will get past basically any protection and you can get access to them for basically $5.

FridayoLeary•13m ago

One minute, what? Can you elaborate on that. I have loads of questions. What exactly were you doing? What consequences did you face? How come you are talking about it?

olalonde•5m ago

> anyone can bypass Cloudflare

How?

sznio•14m ago

There's plenty of DDoS if you're dealing with people petty enough.

The VPS I use will nuke your instance if you run a game server. Not due to resource usage, but because it attracts DDoS like nothing else. Ban a teen for being an asshole and expect your service to be down for a week. And there isn't really Cloudflare for independent game servers. There's Steam Networking but it requires the developer to support it and of course Steam.

Valve's GDC talk about DDoS mitigation for games: https://youtu.be/2CQ1sxPppV4

finghin•53m ago

There’s certainly a business case for “which nines” after the talk of n nines. You ideally want to be available when your competitor, for instance, is not.

carlosjobim•48m ago

Think about this rationally. If Cloudflare doesn't fix it within reasonable time, you can just point to different name servers and have your problem fixed in minutes.

So why be on Cloudflare to start with? Well, if you have a more reliable way then there's no reason. If you have a less reliable way, then you're on average better off with Cloudflare.

erdaltoprak•44m ago

Well I can't change my NS since it's on Cloudflare too but besides that my personal opinion was not about this outage in particular but more the default approach of some websites that don't need all this tech (yes I really was out of groceries)

carlosjobim•41m ago

Is Cloudflare your domain registrar? In that case, yes I think you should think about being less dependent on them.

As for websites which don't need Cloudflare, in my experience almost every website will be DdoS attacked from time to time.

erdaltoprak•30m ago

I am personally really happy with Cloudflare for domains, pages and dns, I don't run critical stuff but some websites are and they should not be lazy about it

darkwater•28m ago

> Is Cloudflare your domain registrar? In that case, yes I think you should think about being less dependent on them.

And why I should overthink my architecture now? If I had to manage redundant systems and keep track of circular dependencies I just could keep managing my infra the old way, no?

I'm being sarcastic here, obviously, but really one of the selling point for cloud back in the day it was "you don't have to care about those details". You just need to care about other details, now.

Semaphor•21m ago

> in my experience almost every website will be DdoS attacked from time to time.

The place I work at has been online since 1996, not even a DoS yet, let alone a DDoS. Though we now use CF to filter all that bot traffic.

karimf•42m ago

It's hard not to use Cloudflare at least for me: good products, "free" for small projects, and if Cloudflare is down no one will blame you since the internet is down.

graemep•34m ago

> if Cloudflare is down no one will blame you since the internet is down.

That is true. it is also the problem. It means the biggest providers do not even need to bother to be reliable because everyone will use them anyway.

reassess_blind•12m ago

Well, no. If they are unreliable to the point of being an outlier when compared to the alternatives then people will switch. At this stage they’re not an outlier.

timeon•29m ago

> if Cloudflare is down no one will blame you since the internet is down.

But this is not really the case. When Azure/AWS were down, same as this one with Cloudflare: significant amount of web was down but most of it was not. It just makes more obvious which provider you use.

Dilettante_•16m ago

"Accountability Sinks"

https://aworkinglibrary.com/writing/accountability-sinks

saxenaabhi•41m ago

Setting up a replica and then pointing your api requests at it when cloudflare request fails is trivial. This way if you have a SPA and as long as your site/app is open the users won't notice.

The issue is DNS since DNS propagation takes time. Does anyone have any ideas here?

isodev•14m ago

Two domains for your api perhaps, a full blown SPA could try one and then the other.

viraptor•11m ago

> Setting up a replica and then pointing your api requests at it when cloudflare request fails is trivial.

Only if you're doing very basic proxy stuff. If you stack multiple features and maybe even start using workers, there may be no 1:1 alternatives to switch to. And definitely not trivially.

tambre•11m ago

Owning your IP space and using Anycast.

numpad0•21m ago

BLOCKCHAINS! I mean, some sort of P2P hosting and/or node discovery would be nice.

nuker•21m ago

> At some point we really need to think if this is the web we want,

You think we have a say in this?

louismerlin•16m ago

You have the power to not host your own infrastructure on aws and behind cloudflare, or in the case of an employer you have the power to fight against the voices arguing for the unsustainable status quo.

reassess_blind•13m ago

If you need DDoS mitigation then you essentially need to rely on a third party. Every third party will have inevitable downtime. For many it’s just whether you’d prefer to be down while everyone else is down or not.

rwky•13m ago

A lot of business would accept the rare downtime from Cloudflare in exchange for the DDoS protection.

If the internet was always a nice place we wouldn't need Cloudflare and similar :(

quaintdev•11m ago

You also have the power to not pay money to big tech

deadbabe•16m ago

Believe me it’s what people want. The alternative is far worse.

severino•4m ago

It's not the web we want, but it's the web corporations want. And everybody else doesn't give a damn.

Andugal•1h ago

OK, it seems to be working again.

kaizenb•1h ago

Nope.

Andugal•58m ago

Indeed, it worked for 2 minutes, but not anymore.

kaizenb•58m ago

Yes.

johnstonnorth•1h ago

https://www.cloudflarestatus.com/incidents/8gmgl950y3h7

dazc•1h ago

Checks Cloudflare Status - yeah, everything's hunky dory bro.

mittermayr•1h ago

I started restarting my own servers thinking something went awry again, that's how much I usually trust them not to be down. Interesting.

mrintegrity•1h ago

Now is as good a time as ever to look at moving our eggs into some other baskets

ilaksh•1h ago

Telnyx seems to be down for me. Actually I lied, I think it is working. at least call connected.

spapas82•1h ago

Cloudflare seems to have degrated performance. Half the requests for my site throw cloudflare 500x errors, the other half work fine.

However the https://www.cloudflarestatus.com/ does not really mention anything relevant. What's the point of having a status page if it lies ?

Update Ah I just checked the status and now I get a big red warning (however the problem existed for like 15 minutes before 11:48 UTC):

> Investigating - Cloudflare is aware of, and investigating an issue which potentially impacts multiple customers. Further detail will be provided as more information becomes available. Nov 18, 2025 - 11:48 UTC

maushu•44m ago

> What's the point of having a status page if it lies ?

Status pages are basically marketing crap right now. The same thing happened with Azure where it took at least 45 minutes to show any change. They can't be trusted.

mrsuprawsm•1h ago

Seems like ChatGPT and Claude are also affected. (CLI Codex still seems to work).

RIP to the engineers fixing this without any AI help.

fullstackchris•48m ago

They better not be using AI to fix this... especially if AI is what caused it! (looking at you, AWS)

tedmundy•1h ago

I think everyone is in the same boat with thinking they took something offline :^)

Concerning though how much the web relies on one (great) service.

pqdbr•53m ago

I had just deployed. Started reverting commits like crazy.

robotfelix•59m ago

> Cloudflare Global Network experiencing issues

> Investigating - Cloudflare is aware of, and investigating an issue which potentially impacts multiple customers. Further detail will be provided as more information becomes available.

Things are back up (a second time) for me.

Cloudflare have updated their status page now to reflect the problems now. It doesn’t sound like they are confident the problem is fully fixed yet.

Edit: and down again a third time!

grrowl•40m ago

it's back again!

TavsiE9s•59m ago

Investigating - Cloudflare is aware of, and investigating an issue which potentially impacts multiple customers. Further detail will be provided as more information becomes available. Nov 18, 2025 - 11:48 UTC

Yeah, those multiple customers is like 70% of the internet.

keshavmunjal•59m ago

Cloudflare is still down

rehanahmed•59m ago

Funny how I couldn't even check on Downdetector.com - because it takes me to a Cloudfare-run captcha, which is now stuck on loading.

The internet is officially down.

keshavmunjal•59m ago

cloudfare down in india

chinathrow•59m ago

It's back up.

brushfoot•58m ago

I've been considering Cloudflare for caching, DDoS protection and WAF, but I don't like furthering the centralization of the Web. And my host (Vultr) has had fantastic uptime over the 10 years I've been on them.

How are others doing this? How is Hacker News hosted/protected?

NicoJuicy•58m ago

Was back up for a moment ( within 5 minutes of being down), currently down again

jjgreen•55m ago

In Yorkshire they would say "up and down like a bride's nightie"

alt227•47m ago

...or a whores drawers

chistev•58m ago

Yea, had trouble accessing Upwork

ea016•57m ago

We're on the enterprise plan, so far we're seeing Dashboard degradation and Turnstile (their captcha service) down. But all proxying/CDN and other services seem to work well.

Semaphor•57m ago

We finally switched to CF a few weeks ago (for bot protection, abusive traffic started getting insane this year), finally we can join in on one of the global outage parties (no cloud usage otherwise, so still more uptime than most).

sutton12345•57m ago

cloudfare is down officially

aap_•57m ago

Crazy to think that it's apparently acceptable to centralize the web like that.

rulehard•57m ago

why does no one use google cloud? literally never has any issues lol

avadhesh18•42m ago

because no one uses it

eruci•56m ago

Searching for Cloudflare alternatives...

CREAMY_BANANA•56m ago

I can't access my fav porn sites, cloudflare Singapore host is down. Oh man!

anuraagvaidya•56m ago

I got an email saying that my OpenAI auto-renewal failed, my credits have run out. I go to OpenAI to reauthorize the card, and I can't login because OpenAI uses Cloudflare for "verifying you are a human" that goes in infinite loop. Great.

Piko•48m ago

oh no, the vibes won't code themselves /s

anuraagvaidya•17m ago

Or even worse, people with apps in production with credits running out

rkwasny•56m ago

downdetector.com is down because it uses cloudflare challenge....

sutton12345•56m ago

do any of users facing cloudfare outage ???!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

vinishbhaskar•55m ago

Tried checking Cloudflare’s status on Downdetector, but Downdetector was also behind Cloudflare. Internet checkmate.

StefanThorpe•55m ago

Whats peoples bets on the root cause of this...?

redrove•48m ago

Their Oregon controlplane somehow? Either a misconfiguration of some sort (BGP??) or a power outage like they had before.

outroll•55m ago

Yep, got around 100 SMSs from our uptime monitoring service that our Cloudflare sites are down. Nothing much we can do but wait.

vldszn•54m ago

x.com is down

aws_eu1•54m ago

ERROR [11:57:30 UTC]: EC2 Launch Failure. Reason: [Security Breach Remediation] Control Plane Metadata Service (IMDS) temporarily offline. System state reports: Dependency integrity check failed (Exit Code 0x80070002). Cannot retrieve authorized kernel image or block device mapping. Termination signal initiated for compromised worker nodes.

ilkkao•54m ago

I can't even load the dashboard to change to "DNS only". Nothing to do?

mak8•54m ago

supabase is down too

bilekas•53m ago

Well it was bound to happen eventually, the "Down Roulette" has decided it should be Cloudflare this week!

aws_eu1•53m ago

CONTROL PLANE FAULT: CRITICAL SECURITY OVERRIDE enforced across us-east-1 and eu-west-2. ERROR CODE: STS.SecurityAuditLockout (403 Forbidden).

chinathrow•52m ago

Context?

jamesnorden•53m ago

Almost like centralizing everything on a single service has consequences.

lpcvoid•53m ago

Can we at some point acknowledge that constant cloud disruptions are too costly, and can we then finally move all of our hosting back on-prem?

donglong•51m ago

can you define "constant"

lpcvoid•48m ago

Well, between AWS US EAST 1 killing half the internet, and this incident, not even a month passed. Meanwhile, my physical servers don't care and happily serve many people at a cheaper cost than any cloud offer.

alt227•47m ago

We had an Azure outage in between those 2 as well.

chistev•46m ago

How do you back up?

lpcvoid•35m ago

We have a few colocated servers offsite, each in a different region, each with a zpool of mirrored spinning rust. We use rsync across those at different times.

donglong•45m ago

never build on us-east-1, everyone knows that ;)

vlovich123•42m ago

You realize these are two different companies right? If you’re saying “I’m an AWS customer with cloudflare in front” I think you’ve failed to realize that two 99.9% available services in series have a combined availability of ~99.8% - that’s just math.

Your physical servers should have similar issues if you put a CDN in front unless the physical server is able to achieve a 100% uptime (100% * 3 9s = 3 9s). Or you don’t have a CDN but can be trivially knocked offline by the tiniest botnet (or even hitting hacker news front page)

lpcvoid•37m ago

I do. But I put both into the "cloud offering off-prem for very much money" shoebox. I setup a CDN once using VPS from different hosting providers for under 100 USD a month, which I would vastly prefer over trusting anything cloud.

And yes, I know that there's sites that need the scale of an operation like Cloudflare or AWS. But 99.9(...)% of pages don't, and people should start realizing that.

darkwater•50m ago

Funnily and ironically enough, I was trying to check out a few things on Ansible Galaxy and... I ended up here trying to submit the link for the CF ongoing incident

DC-3•50m ago

It's the old IBM thing. If your website goes down along with everyone else's because of Cloudflare, you shrug and say "nothing we could do, we were following the industry standard". If your website goes down because of on-prem then it's very much your problem and maybe you get to look forward to an exciting debrief with your manager's manager.

lpcvoid•47m ago

That's lazy engineering and I don't think we as technical, rational people should make that our way of working. I know the saying, but I disagree with it. My fuckups, my problem, but at least I can avoid fuckups actively if I am in charge.

reassess_blind•46m ago

How do you mitigate large scale DDoS?

lpcvoid•42m ago

I don't, since my stuff is reachable only within the company network/VPN. If I needed to though, I would consult the BSI list of official DDOS mitigation services [0] and evaluate each one before deciding. I would not auto-pick Cloudflare.

[0] (German) https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Cyber-Si...

reassess_blind•27m ago

When the solution you pick inevitably has downtime too you’re in the same boat.

DDoS mitigation is one of those areas that an on-prem solution just isn’t well suited to solve.

lpcvoid•22m ago

Yeah, but people aren't using Cloudflare just for DDOS Mitigation. Some are running pretty much everything over it, from DNS to edge caching to load balancing and even hosting. That's what I oppose mainly.

saubeidl•43m ago

The problem is the people that sign our checks usually aren't technical, rational people.

The system isn't designed for technical, rational decision making.

lpcvoid•40m ago

That's fair, yeah, and I agree it's not always feasible - but if you have any influence over technical direction at your org, I encourage what I wrote above. Otherwise yeah, let the pea counters in the C-Levels dig their own grave.

rkangel•33m ago

I would only consider doing stuff on-prem because of services like Cloudflare. You can have some of the global features like edge-caching while also getting the (cost) benefits of on-prem.

celltalk•53m ago

I am glad my personal site is not affected, what would I do without all those incoming traffic.

ashsk0110•53m ago

World infrastructure is taking a hit. First us-east and now this.

thoughtogram•53m ago

Cant websites have an auto backup/redirect in case cloudflare or AWS go down?

mberger•52m ago

Is it DNS? I went to check the isitdns.com but got a cloudflare error

ablation•52m ago

It's knocked out Turnstile too, which means I can't even log in to my Cloudflare dash to bypass my site's proxying via Cloudflare.

porker•47m ago

Even if you could, the DNS entries aren't loading. And then the page 404's.

stavros•44m ago

Oh you aren't missing much, the dashboard doesn't load anyway.

bamboozled•52m ago

This one is huge.

drchiu•51m ago

Just when the eastern cities are waking up too.

aws_eu1•51m ago

ERROR [12:00:21 UTC]: CF_EDGE_ROUTING_FAILURE. Reason: Origin-Shield connectivity loss detected within multi-region fabric. BGP path withdrawal initiated for critical LCP clusters (LCP-LON, LCP-FRA). Status code 521/522 flood reported globally. Geo-location failover services degraded. DNS resolution timeout on 1.1.1.1/1.0.0.1. Traffic flow re-routing pending verification of internal control plane integrity.

dontdoxxme•16m ago

Did you ask an LLM to try to guess the error message?

jamesnorden•51m ago

And no lesson about single point of failure and centralization was learned that day.

carlosjobim•45m ago

Where is the single point of failure? You can point to different name servers and swiftly remove Cloudflare from your setup.

JustSkyfall•43m ago

Not if you’re using Workers/Pages!

blackjackfoe•42m ago

If your site is only hosted on one server and it catches fire, you can swiftly reinstall on a new server and change the IP your domain is pointing to, too... Still a single point of failure.

carlosjobim•35m ago

Yes, everything in the world is a single point of failure and has always been, if we look at things that way. But if it can be remedied quickly, then it's not a huge concern.

ashed96•51m ago

Suddenly feeling better about our 99.9% uptime SLA.

When even Cloudflare goes down, nobody can blame the little guys.

udev4096•51m ago

Would be funny if it was a record breaking ddos on cloudflare

KabuTheSilicon•42m ago

The reason why you laugh

lofaszvanitt•51m ago

Seemingly nobody cares about being in two different availability zones. Or is this a deeper problem?

whd5015•51m ago

just scheduled maintenance in Tahiti guys, nothing to see here

rikafurude21•44m ago

could be related? Tahiti is a small island in the south pacific, seems rather suspicious

demontime•51m ago

Twitter too is down, almost half the internet

MaximilianEmel•51m ago

When's the last time Cloudflare had such an outage?

Metacelsus•51m ago

Yeah I just got a 500 error on medRxiv

akashvekariya•51m ago

We are doomed! Is it another vibe-coding disaster?

robertwt7•51m ago

this should affect a lot sites? I'm trying to access tailwindcss and I can't as well!

chistev•47m ago

Not my site though

https://www.rxjourney.net/

csomar•46m ago

I am using cloudflare as back-end for my site (workers) but have disabled all their other offerings. I was affected for a short while but seems to be less affected than other people.

biinjo•38m ago

Lol. I mean I love Tailwind but it seems like the least trivial site/service to be down right now haha.

ImScope052300•50m ago

Dude what's up with X???

kmarc•50m ago

Yesterday I decided to finally write my makefiles to "mirror" (make available offline) the docs of the libraries I'm using. doc2dash for sphinx-enabled projects, and then using dash / zeal.

Then I was like... "when did I last time fly for 10+ hours and wanted to do programming, etc, so that I need offline docs?" So I gave up.

Today I can't browse the libs' docs quickly, so I'm resuming the work on my local mirroring :-)

ImScope052300•50m ago

Bro what's up with X???

akashvekariya•50m ago

We're doomed. Is this another vibe-coding disaster?

lambdaba•50m ago

it's funny I first noticed this visiting a random blog, then went on X and got the same error... is Cloudflare the Internet now?

fullstackchris•50m ago

Supabase is down bad too... need to work on my project!

fullstackchris•44m ago

Haha they updated their status page: "Identified - A global upstream provider is currently experiencing an outage which is impacting platform-level and project-level services"

A global upstream provider :)

farhadhf•50m ago

Pretty much everything is down (checking from the Netherlands). The Cloudflare dashboard itself is experiencing an outage as well.

Not-so-funny thing is that the Betterstack dashboard is down but our status page hosted by Betterstack is up, and we can't access the dashboard to create an incident and let our customers know what's going on.

Edit: wording.

ablation•48m ago

This is a big one.

csomar•41m ago

Seems like workers are less affected and maybe betterstack has decided to bypass cloudflare "stuff" for the status pages? (maybe to cut down costs). My site is still up though some GitHub runners did show it failed at certain points.

tyingq•39m ago

I have a workers + kv app that seems fine right now.

csomar•36m ago

Pretty sure they went down for a while because I have 4xx errors they returned but apparently it was short-lived. I wonder if their workers infra. failed for a moment and that let to a total collapse of all of their products?

biinjo•41m ago

Same here. We’re using OhDear. The status page is available but I can’t post an incident because their service is also behind Cloudflare.

Mojah•32m ago

Co-founder here, we'll be working on better ways to handle this over the coming days.

pell•40m ago

It’s that time of the year again where we all realize that relying on AWS and Cloudflare to this degree is pretty dangerous but then again it’s difficult to switch at this point.

If there is a slight positive note to all this, then it is that these outages are so large that customers usually seem to be quite understanding.

tommica•38m ago

> It’s that time of the year again

It's monthly by now

fusl•37m ago

Happy to hear anyone's suggestions about where else to go or what else to do in regards to protecting from large-scale volumetric DDoS attacks. Pretty much every CDN provider nowadays has stacked up enough capacity to tank these kind of attacks, good luck trying to combat these yourself these days?

isodev•32m ago

Anubis and/or Bunny are good alternatives/combination depending on your exact needs

- https://anubis.techaro.lol/

- https://bunny.net/

fusl•28m ago

Unfortunately Anubis doesn't help where my pipe to the internet isn't fat enough to just eat up all the bandwidth that the attacker has available. Renting tens of terabits of capacity isn't cheap and DDoS attacks nowadays are in the scale of that. BunnyCDN's DDoS protection is unfortunately too basic to filter out anything that's ever so slightly more sophisticated. Cloudflare's flexibility in terms of custom rulesets and their global pre-trained rulesets (based on attacks they've seen in the past) is imo just unbeatable at this time.

isodev•25m ago

The Bunny Shield is quite similar to the Cloudflare setup. Maybe not 100% overlap of features but unless you’re Twitter or Facebook, it’s probably enough.

I think at the very least, one should plan the ability to switch to an alternative when your main choice fails… which together with AWS and GitHub is a weekly event now.

lbreakjai•36m ago

If I choose AWS/cloudflare and we're down with half of the internet, then I don't even need to explain it to my boss' bosses, because there will be an article in the mainstream media.

If I choose something else, we're down, and our competitors aren't, then my overlords will start asking a lot of questions.

jfengel•23m ago

And if everyone else is down, and you are not, you will get no credit.

stevepotter•20m ago

Yup. AWS went down at a previous job and everyone basically took the day off and the company collectively chuckled. Cloudflare is interesting because most execs don’t know about it so I’d imagine they’d be less forgiving. “So what does cloudflare do for us exactly? Don’t we already have aws?”

timeon•14m ago

In reality it is not half of the internet. That is just marketing. I've personally noticed one news site while others were working. And I guess sites like that will get the blame.

weird-eye-issue•36m ago

Oh no, we had 30 minutes of downtime this year :(

pell•33m ago

I do think this is tenable as long as these services are reliable. Even though there have been some outages I would argue that they’re incredibly reliable at this point. If though this ever changes the costs to move to a competitor won’t be as simple as pushing a repository elsewhere, especially for AWS. I think that’s where some of the potential danger lies.

isodev•35m ago

Unless you’re say at airport trying to file a luggage claim … or at the pharmacy trying to get your prescription. I think as a community we have a responsibility to do better than this.

sigilis•21m ago

You aren’t cloudflare’s customer in these examples. It depends on the companies that are actually paying for and using the service to complain. Odds are that they won’t care on your behalf due to how our society is structured.

Not really sure how our community is supposed to deal with this.

isodev•13m ago

“We” are the ones making the architecture and the technical specs of these services. Taking care for it to still work when your favourite FAANGMC is down seems like something we can help with.

esskay•39m ago

When its back up, do yourself a favour and rent a $5/mo vps in another country from a provider like OVH or Hetzner and stick your status page on that.

"Yes but what if they go down" - it doesnt matter, having it hosted by someone who can be down for the same reason as your main product/service is a recipe for disaster.

hcaz•39m ago

https://cachethq.io/ is great for this

fusl•33m ago

Been using Cachet for quite a while before inevitably migrating to Atlassian's Statuspage.io. I'm a huge fan of self-hosting and self-managing every single thing in existence but Cachet was just such a PITA to maintain and there was just no other good alternative to Cachet that was also open source.

jwr•26m ago

Amusingly enough, it is down right now because of Cloudflare :-)

jwr•26m ago

Or use a service like https://updown.io/ (I host my status page there).

postalcoder•37m ago

Refresh your dashboard constantly in multiple tabs. Cloudflare dashboard is down-ish, not totally down. If you're persistent you can turn off the turnstile and proxy.

It took a few minutes but I got https://hcker.news off of it.

farhadhf•31m ago

I'm already logged in on the cloudflare dashboard and trying to disable the CF proxy, but getting "404 | Either this page does not exist, or you do not have permission to access it" when trying to access the DNS configuration page.

postalcoder•30m ago

Keep refreshing, I got that too!

skywhopper•6m ago

Not saying not to do this to get through, but just as an observation, it’s also the sort of thing that can make these issues a nightmare to remediate, since the outage can actually draw more traffic just as things are warming up, from customers desperate to get through.

But then, that’s what Cloudflare signed up to be.

celltalk•31m ago

I think there is a big business opportunity here. Make a site that let companies put their status update on local vps for $100.

ramon156•30m ago

on-demand status balancing!

colinbartlett•28m ago

Even the Cloudflare status page, hosted by Atlassian Statuspage, is suffering. Probably due to the traffic crush.

nrhrjrjrjtntbt•27m ago

Status pigeons.

chrisandchris•26m ago

I don't get why you need such a service for a status page with 99.whatever% uptime. I mean, your status page only has to be up if everything else is down, so maybe 1% uptime is fine.

nnyms1•49m ago

I got several emails from some uptime monitors I setup due to failing checks on my website and funnily enough I cannot log into any of them.

BetterStack, InStatus and HetrixTools seemingly all use Cloudflare on their dashboards, which means I can't login but I keep getting "your website/API is down" emails.

Update: I also can't login to UptimeRobot and Pulsetic. Now, I am getting seriously concerned about the sheer degree of centralization we have for CDNs/login turnstiles on Cloudflare.

chromehearts•49m ago

And here I was wondering why my website shut down & why I couldn't tweet about it

prityush14•49m ago

when ig went down, I came to X to post Now that X is down, is hn the place to be

m11a•49m ago

So ... any bets the cause isn't DNS?

reassess_blind•49m ago

Who wants to join me at the Winchester for a pint, and wait for this all to blow over?

Dilettante_•8m ago

Got some red on you...

BilalBudhani•49m ago

everything is down except HN :D

jsheard•46m ago

The just-one-big-server-in-someones-basement stack remains undefeated.

gjsman-1000•42m ago

I don’t know; HN historically has had way worse uptime than Cloudflare.

mambru•37m ago

Except it isn't that big?

allanmacgregor•42m ago

Digg.com is working perfectly hahahaha

mrweasel•25m ago

It's a little surprising how little it affect me. I believe it's around 20% of websites that use CloudFlare in some form or another.

kgwxd•48m ago

My Window System seems to be working fine.

taneq•48m ago

Aw man, how dare this affect me personally? :P (Tried to get to openstreetmap.org which is behind cloudflare.)

baccatore•48m ago

Tokyo too

binanabread•48m ago

aaa the only night im free to watch nana and cloudfare sabotages me T_T

chistev•47m ago

How come HN is never down with all these outages?

meeb•47m ago

Because HN doesn't use Cloudflare.

chistev•45m ago

Also doesn't use AWS or Azure because it didn't go down with them either.

us0r•34m ago

I thought the same until I started looking:

Name Server: NS-225.AWSDNS-28.COM Name Server: NS-1411.AWSDNS-48.ORG Name Server: NS-1914.AWSDNS-47.CO.UK Name Server: NS-556.AWSDNS-05.NET

At least for DNS. Data center appears to be Lightedge.

Bender•25m ago

HN sometimes rarely is pointed to AWS when under attack.

ardfard•44m ago

Does HN self-host too?

ArcHound•40m ago

I'd love to read an article describing the HN setup. Seems that they got a lot of things right - self registration, influx of people during outages and plenty others. Admins, if you see this, please write about your craft!

Akronymus•37m ago

I am pretty sure @dang wrote about it on here before, but I can't seem to find it.

b2ccb2•31m ago

Here https://news.ycombinator.com/item?id=28478379 and https://news.ycombinator.com/item?id=27452276

ArcHound•20m ago

Thanks! Seems that's what we'll get as there are apparently serious money which can be made by knowing the guts of HN as per https://news.ycombinator.com/item?id=27454354.

ArcHound•29m ago

Seems like so, these submissions from them look related, but not quite it:

- [Sorry I broke the server](https://news.ycombinator.com/item?id=9052128) - [New attempt at mobile markup](https://news.ycombinator.com/item?id=10489499) - [Clickable domains and QoL](https://news.ycombinator.com/item?id=10223645) - [New features and a moderator](https://news.ycombinator.com/item?id=12073675) - [Thanks to thehodge and littlewarden, this site is up today](https://news.ycombinator.com/item?id=28472350)

Maybe one day. Seeing all of these big providers stumbling, an article about HN staying on top of everything would surely resonate.

Bender•44m ago

HN is just one active and one standby server at M5 Computer Security running BSD.

pmlnr•43m ago

There are things out there which are running from a bare metal host, without relying on someone else's computer (aka the cloud). HN is one of them.

duckerduck•41m ago

They stopped using Cloudflare some time ago

https://news.ycombinator.com/item?id=18188832

rwaksmunski•37m ago

FreeBSD on bare metal hooked up to a nice network.

philipwhiuk•34m ago

HN is running on the server the rest of the cloud rents time from.

The outages are the Roomba.

materikzen•47m ago

facing cloudflare downtime

gbil•47m ago

The irony is that if you follow the relevant link [1]in the error page , you get this

> If the problem isn’t resolved in the next few minutes, it’s most likely an issue with the web server you were trying to reach.

[1] https://www.cloudflare.com/5xx-error-landing/?utm_source=err...

gbajson•47m ago

Poland. Most of the popular sites are down. Including community forum on Cloudflare.

djsjajah•47m ago

I went to check how many services are being impacted on down detector, but it was down.

greatgib•47m ago

What is funny us that on their global status list for services, everything looks green except "network" that is "offline".

virajk_31•47m ago

I was shouting at network guy/colleague, how come challenges.cloudflare.com got blocked!! damn, I must apologise to him.

yipbub•45m ago

Probably better not to shout in the first place.

virajk_31•41m ago

It was friendly fire, nothing serious. haha

weird-eye-issue•35m ago

> friendly fire

Yeah I don't think you are using this phrase correctly

virajk_31•32m ago

Just trying to fit the jargons...

enidjv•38m ago

Don't worry beer gonna fix everything

uniq7•36m ago

Even if he blocked it by accident, that is not a reason to shout.

Shouting will not prevent errors, and you are only creating a hostile work environment where not acting is better than the risk of making a mistake and triggering an aggressive response from your part.

virajk_31•26m ago

It wasn't aggressive exchange, but will definitely consider your comment.

Fiveplus•22m ago

Please don't yell at your colleagues.

KabuTheSilicon•47m ago

akxeder.eth.ac is working

rvz•47m ago

Probably a good time to contact the CEO of Cloudflare.

Looking forward to the post-mortem.

dodyg•46m ago

chatgpt.com is not working because they are relying on cloudflare for challenges

conradfr•46m ago

I thought I would be clever by switching domain endpoints from proxied to dns but Cloudflare admin page is also not working correctly ;)

edit: it's up!

buybackoff•46m ago

The main bike rental Velib in Paris has the app not working, but the bikes can be taken with NFC. However, my station, which is always full at this time, is now empty, with only 2 bad bikes. It maybe related. Yet, push notifications are working.

I'm going to take the metro now and thinking how long do we have until the entire transit network goes down because of a similar incident.

varshneydevansh•46m ago

I was about to cream watching my fav video on X and it is down

donglong•46m ago

genuinely makes me sad for the people there. this must be a living nightmare right now.

vntok•43m ago

Why? If any company has enough technical people, resources & processes in place it must be them, no?

finlayy•46m ago

Ironic, cloudflare taking downdetector with it

miiniiiie•45m ago

Feels like 25% of the Internet is down just because of fuckin' cloudflare.

I'm leaving the redaction because I couldn't work atm...

Time for a beer , greetings from germany!

dmezzetti•45m ago

This one is bigger than the AWS East outage...

finlayy•45m ago

Ironic, Cloudflare taking DownDetector with it

Odablock•45m ago

Forced to play Runescape now

GooniestGoon•45m ago

Bruh I was watching porn on Twitter, I thought the FBI got my ahh.

KabuTheSilicon•44m ago

akxeder.eth.ac is not down

neya•44m ago

The biggest learning for me from this incident - NEVER make your DNS provider and CDN provider the same vendor. Now, I can't login into the dashboard, even to switch the DNS. Sigh.

rollulus•44m ago

Classic. I see issues. Vendor’s status page is all green. Go to HN to find the confirmation. Applies to AWS, GH, everyone.

Edit: beautiful, this decentralised design of the internet.

markild•42m ago

I get the feeling that all "serious" businesses have manual processes for publicly facing status pages, for political reasons.

I don't like it.

bnjm•31m ago

SLA breaches have consequences, no big conspiracy there

markild•20m ago

Not at all saying it's a conspiracy, I just think it's a lack of transparency.

I get why, but it would give me more confidence if they would tell me about everything.

skywhopper•4m ago

At some level, the status updates have to be manual. Any automation you try to build on top is inevitably going to break in a crisis situation.

miiniiiie•44m ago

Couldn't work. Fuckin' cloudflare . Feels like 25% of the Internet is down.

Im going home. Time for a beer .

Greetings from germany

saubeidl•44m ago

Cloud in general was a mistake. We took a system explicitly designed for decentralization and resilience and centralized it and created a few neat points of failure to take the whole damn thing down.

is_true•36m ago

It took me a while to understand it, but the beauty of it is that when it fails, lot of things fail.

Almost no one gets mad if your site and half the internet were down.

saubeidl•27m ago

Sure, but that is also a giant weakness. Say in a future conflict with Russia or China, or hell, even North Korea.

They'd only have to take down a few services to completely cripple the West - the exact case ARPANET was designed to prevent.

mosselman•34m ago

Cloudflare provides some nice services that have nothing to do with cloud or not. You can self-host private tunnels, application firewalls, traffic filtering, etc, or you can focus on building your application and managing your servers.

I am a self-host enthousiast. So I use Hetzner, Kamal and other tools for self-managing our servers, but we still have Cloudflare in front of them because we didn't want to handle the parts I mentioned (yet, we might sometime).

Calling it a mistake is a very narrow look at it. Just because it goes down every now and then, it isn't a mistake. Going for cloud or not has its trade-offs and I agree that paying 200 dollars a month for a 1GB Heroku Redis instance is complete madness when you can get a 4GB VPS on Hetzner for 3,8 a month. Then again, some people are willing to make that trade-off for not having to manage the servers.

Cloud servers have taught me so much about working with servers because they are so easy and cheap to spin up, experiment with and then get rid of again. If I had had to buy racks and host them each time I wanted to try something, I would've never done it.

locallost•21m ago

Thanks, I was too lazy to write this, and noticed this comment multiple times now. It's good to be sceptical at times, but in this case it simply misses the mark.

saubeidl•12m ago

Sure, it's a great fair-weather technology, makes some things cheap and easy.

But in the face of adversity, it's a huge liability. Imagine Chinese Hackers taking down AWS, Cloudflare, Azure and GCP simultaneously in some future conflict. Imagine what that would do to the West.

I don't believe in Fukuyamas End of History. History is still happening, and the choices we make will determine how it plays out.

nijave•21m ago

Threat actors (DDoS) and AI scraping already threw a wrench in decentralization. It's become quite difficult to host anything even marginally popular without robust infrastructure that can eat a lot of traffic

firecall•44m ago

10.30pm here in Australia...

and my alarms are going off my and support line is ringing...

I cant even login to my CF dashboard to disable the CDN!

Edit... and it's back. Hopefully it will stay up!

jpdus•43m ago

germany as well. Claude down too

Culonavirus•23m ago

X, Chatgpt, all kinds of sites and services around the eu, it's a massive outage

ReedorReed•43m ago

There is an election in Denmark today, I wonder if this will affect that. The governments website is not accessible at the moment because it uses Cloudflare.

orangeyjuicey•43m ago

So do we have a guarantee that posts are not made by AI for a few minutes?

tzpbingo•43m ago

20% of websites worldwide are down.

ramon156•38m ago

source? would love to see

ed_mercer•43m ago

I got an invoice from them right before the outage. Hopefully when they restore everything, they'll have forgotten about it!

thearnabsarkar•43m ago

Did something happen to the Cloudflare lava lamps ?

Brajeshwar•42m ago

Ah! Well, all of my websites are down! I’m going to take screenshots and have it as part of my Time Capsule Album, “Once upon a Time, my websites used to go down.”

KabuTheSilicon•34m ago

Once upon a time, the end of the world happened

jameslk•42m ago

Cloudflare's own status page is not responding. I guess it's down too?

udev4096•42m ago

A good reminder for advancing decentralization and p2p networks!

cipixul•42m ago

Bucharest Cloudflare down too

StefanThorpe•42m ago

Strange thing is this is in multiple CD regions all using bot & WAF are down, just got a colueuge to check our site and both London & Singapour cloudflare servers are out... And I cant even login to the cloudflare dash to re-route critical traffic . Likely this is accidental, but one day there will be something malicous that will have big impacts with how centralised the internet now is.

clownelon•42m ago

Who is laughing now Elon ?

haik90•42m ago

It is a relief that they hosted the status page on someone else's infrastructure.

3xstphvs•42m ago

Australia here. plenty down rn

cf_py•41m ago

Didn't have my site on cloudflare bc it would be faster for chinese users (its main demographic) so i THOUGHT i was fine for a second until i remembered the data storage api is behind cloudflare

cipixul•41m ago

claude.ai down too... lots of programmers are gonna have to pretend they code in another way...

wesrobin•37m ago

It's working fine for me

cipixul•22m ago

Yep, seems to work cause my nearby colleague started copy/pasting massive chunks of code again.

I guess claude is more important than your average site :)

nrhrjrjrjtntbt•22m ago

Easy: "the site is done, it is fantastic, but cloudflare is down so you cant see it"

sharathnryn•41m ago

the sheer number of websites this is taken down!

testingofindia•41m ago

even popular apps like x.com, chatgpt.com are down.

tebbers•41m ago

My uptime monitor OnlineOrNot is also down...

rozenmd•29m ago

OnlineOrNot's fallen back to AWS for monitoring, so you should still be getting alerts.

The dashboard's API server runs on Cloudflare and is currently blocking all logins, will fix.

testingofindia•40m ago

even the famous applications like Chatgpt, x.com are down

forthaven•40m ago

I'm betting on DNS fail

ChewbyP•40m ago

Just happy I wasnt IP banned lol

forthaven•40m ago

I'm betting on another DNS failure

raxxorraxor•40m ago

Seems to work again. 40min downtime for many services it seems.

pc2g4d•40m ago

I was reading up on home lab server racks, and every single site is down with a Cloudflare error. So much for DIY!

tristanperry•39m ago

ChatGPT and Claude are down as a result, too.

Y'know, along with most other SAAS services.

miiniiiie•39m ago

ELON! GO AND KICK THOSE CLOUDFLARE ASSES!

or search a new job for yourself. Maybe digging to the earth core. Why? Idk. Because then you can say: I did it, or so.

crazyooly•39m ago

We really do have two surprise holidays every year: AWS Day and Cloudflare Day. Happy outages, everyone.

3xstphvs•39m ago

it works, then stops, then works

MrPasancalla•39m ago

Just when I was assigned a task yesterday but decided to do it today early morning.

sansiro9•38m ago

Oh, look! Cloudflare is down. Let's check down detector to make sure it's not just me > Downdetector is using Cloudflare captcha. Yep, it's down.

Tixx7•37m ago

some sites are already up again, including the cf dash and downdetector, both ironically down a few minutes ago

Woods369•37m ago

syntax error: unexpected semicolon

fungungun•37m ago

its back !

adrianhacar•37m ago

The sites I host on Cloudflare are all down. Also, even ChatGPT was down for a while, showing the error: "Please unblock challenges.cloudflare.com to proceed."

dsco•21m ago

This is still the case for me

pelagicAustral•37m ago

Things seem to be coming back up... been almost 45 minutes, since my first alert came at 0836

vednig•36m ago

hacked.stream was down too

philipwhiuk•35m ago

Looks like the status page is suffering too because it can't load jQuery:

(index):64 Uncaught ReferenceError: $ is not defined at (index):64:3

iamawacko•35m ago

Wow, so much is down. Nothing Cloudflare protected is loading for me in Indiana, and the Cloudflare dashboard is broken as well.

I hope it gets resolved in the next hour or two, or it could be a serious problem for me.

vxworkss•34m ago

Glad to see things are actually working here! Also, my website (halomate.ai) is using CF too, and surprisingly, it's working fine as well

duncans•24m ago

You spoke too soon!

nhatcher•34m ago

Off topic, but the 500 page from prusa3d is quite good:

https://www.prusa3d.com/

https://imgur.com/a/OW5KL8r

mig4ng•34m ago

Cloudflare Dashboard/Clicky clicky UI is down. I really appreciate that their API is still working. Small change in our Terraform configuration and now I can go lunch in peace knowing our clients at skeeled can keep working if wanted:

resource "cloudflare_dns_record"

- proxied = true

+ proxied = false

NicoJuicy•34m ago

Seems to be back up

elcapitan•32m ago

Black HN ribbon for the Internet

tristanperry•26m ago

I assume you're joking, but as an FYI, Rebecca Heineman died:

https://www.pcgamer.com/gaming-industry/legendary-game-desig...

theideaofcoffee•31m ago

Love that for them. Congrats on building such a re-centralized web!

rvz•14m ago

Cloudflare is now a systemic risk for a state-sponsored attacker to bring down the entire web.

ntoskrnl_exe•31m ago

It’s been 45 minutes and I’m already looking forward to the day Kevin Fang makes a video about this

tgv•30m ago

Seems to be over.

ravikinhajaat•30m ago

anyone have reference for flutter opening +917494920753

mittermayr•28m ago

Even Cloudflare Status is now down, oh boy :) https://postimg.cc/LJVKYmks

carter-0•26m ago

Even your postimg.cc link is down for me.. (at least their CSS is)

https://ibb.co/QF6X0pX9

hirako2000•14m ago

Postimg's CDN is down

wazoox•28m ago

I discovered the problem by trying to access https://downforeveryoneorjustme.com/ ironically :)

martiuk•28m ago

API still seems to work if you already have a script to hand to unproxy everything.

kumarski•28m ago

I'm going to buy more.

chid•25m ago

Took quite a while for it to show up oddly.

joemonsters•25m ago

Now I can switch everything off and go home. We are not using CF at our site, but CF error it is a good reason to have a day off

dheatov•25m ago

Seems like coudflare activated the maximum llm-scraper-bot-protection for everyone.

digid8a•24m ago

just yesterday cloudflare announced it was acquiring replicate (ai to "help" it's workers) i believe

nrhrjrjrjtntbt•24m ago

You

Browser Working

San Jose

Cloudflare Error

mysite.com

Host Working

Lol! Like a solar eclipse!

bpavuk•23m ago

Ukraine. Sporadic outages as well. Error pages blame Cloudflare Warsaw servers.

hamish-b•22m ago

What a wild ride, the traffic to my site is more akin to a rollercoaster. Got better for a few mins and then fell back apart.

kosolam•22m ago

Still ongoing. Some requests going through. Some get the cf error page.

Fiveplus•22m ago

Cloudflare is the real backbone of the internet in 2025. It should be a globalized property like ICANN or something

aeve890•4m ago

How would that prevent outages? Honest question

ahmedhossamdev•22m ago

lol

dheatov•22m ago

Reminder that this is not the web we want.

seydor•21m ago

More proof that central planning doesn't work

sidcool•21m ago

Even Twitter is down. Most of my customers are shouting at the top of their head!

sidcool•21m ago

The top black bar is appropriate /s

Fiveplus•21m ago

Even twitter is gone. Where will I post memes mocking cloudflare?

gethly•20m ago

Ha ha ha hahahahahaaaa hahahahahaaaahahaha, fuck 'em.

Dilettante_•20m ago

  Our support portal provider is currently experiencing issues

Are they using Cloudflare perchance? (scnr)

import•19m ago

Funny that their status page shows almost all locations “Operational” but they’re not. Are they updating the page manually and keep it green?

lilppdavidson•18m ago

I’m assuming hard rock (bet) is run by cloudflare also

azholder•18m ago

what's a good alternative for their WAF, that isn't enterprise expensive?

nrhrjrjrjtntbt•18m ago

Is it DNS or BGP?

hypeatei•18m ago

Why do people use the reverse proxy functionality of Cloudflare? I've worked at small to medium sized businesses that never had any of this while running public facing websites and they were/are just fine.

Same goes for my personal projects: I've never been worried about being targeted by a botnet so much that I introduce a single point of failure like this.

xyzzy9563•12m ago

It gives really good caching functionality so you can have large amounts of traffic and your site can easily handle it. Plus they don't charge for egress traffic.

arccy•12m ago

people think that running nodejs servers are a good idea, and those fall over if there's ever so much as a stiff breeze, so they put cloudflare in front and call it a day.

lofaszvanitt•11m ago

It's chic. Young bois or adult pepl with boi like mentality.

What, they have Cloudflare and we don't? We also must have cloudflare. Don't ask why.

Now that you have it, you are at least level 15 and not a peasant.

Same applies to every braindead framework on the web. The gadget mind of the bois is the cause for all this.

digid8a•17m ago

just yesterday cloudflare announced it was acquiring replicate (ai platform) "the Workers Platform mission: Our goal all along has been to enable developers to build full-stack applications without having to burden themselves with infrastructure" according to cloudflare's blog, are we cooked?

lilppdavidson•17m ago

I’m assuming Hard Rock (Bet) is run by Cloudflare too

azholder•17m ago

Lots of valid concern about us all using CF, but is their an alternative to their WAF that isn't enterprise expensive?

Fiveplus•16m ago

I asked google what's an alternative to Cloduflare. It says, "A complete list of Cloudflare alternatives depends on which specific service (CDN, security, Zero Trust, edge computing, etc.) you are replacing, as no single competitor offers the exact same all-in-one suite"

Fiveplus•13m ago

I would love to be a bee on the wall in the room where Cloudflare response engineers are working right now.

littlecranky67•12m ago

AWS, then Azure, now Cloudflare. Welcome to the AI era. Meanwhile my hetzner vServer has been running for three years without issues.

yeehaw125•12m ago

The irony of being in the middle of reading how Basecamp got off the cloud and the external link being down with a CF error :D

EvanAnderson•10m ago

We've traded DDoS for centralized DoS.

rikafurude21•7m ago

centralized incompetency causes distributed denial of service

mintflow•7m ago

Recently my multiple VPN server nodes just randomly cannot connect to cloudflare CDN IPs, from different provider VPS, while the Host Linux network does not have the issue; vpp share the same address with Linux and use tc stateless NAT to do the trick.

I finally work around this by change the tcp options sent by vpp tcp stack.

But the whole thing made me worry there must be something deployed which cause this issue.

But I do not think that related with this network issue, it just reminds me the above, I feel there are frequently new articles about cloudflare networking, maybe new method or new deployment sort of related high probability of issues

rco8786•6m ago

Is it me or has there been a very noticeable uptick in large scale infra-level outages lately? AWS, Cloudflare, etc have all been way under whatever SLA they publish.

alt227•3m ago

For me the only silver lining to all these cloud outages is now we know that their published SLA times mean absolutely nothing. The number of 9's used to at least give an indication of intent of reliability, now they are twisted to whatever metric the company wants to represent and dont actually represent guaranteed uptime anywhere.

agile-gift0262•5m ago

In the beginning I thought my IP fell on the wrong side of Cloudflare and thought I was being blocked from ~80% of the internet. I was starting to panic

BatteryMountain•3m ago

What have you been looking at citizen?

BatteryMountain•4m ago

Whole bunch of local South African sites are dead, with cloudflare http 500 errors. Can see Lisbon & Amsterdam crashing out.

BrouteMinou•4m ago

Do you remember when the Internet was redundant and resilient?

It seems 20% of the Internet is down every two weeks now.

KabuTheSilicon•3m ago

70% of the internet is down

chakintosh•4m ago

The whole internet hinges on this one company

turnsout•4m ago

This is reason 1, 2 and 3 on my "Top 3 Reasons to not Put All Eggs in One Basket" list.

Multiple Digital Ocean services down

Kerala becomes first state in India to eradicate extreme poverty

It's not surprising that 95% of AI enterprise projects fail

Towards interplanetary QUIC traffic with Rust Quinn

Google Gemini 3 Pro Model Card [pdf]

The Ochre Origins of Art

Show HN: Filtered GitHub Trends

SplitFlow: Flow Decomposition for Inversion-Free Text-to-Image Editing

GoSign Desktop RCE flaws affecting users in Italy

Sustaining Open Source: The Next 25 Years Depend on What We Do Together Now

Ask HN: Why is there black line on top? I guess not because of Cloudflare outage

Mathematics and Computation (2019) [pdf]

Apakah WhatsApp Bank Permata 24 Jam

Avi Wigderson – P vs. NP [video]

Do We Need to Slash the Debt?

The nature of the Theory of Computation (2018) [pdf]

Ask HN: Does insurance cover payouts during a global outage?

A DST primer for unit test maxxers

Show HN: Tool to check risks of your startup

Technoblogy – USB-C Power Delivery Dongle

Open Source in Focus: Projects We're Proud to Support

Show HN: An open source static site that parses and visualizes log files locally

MaggieLab – Simple, non-destructive, online image editor

Open Source Power

Show HN: SecuriScan – Open-source Chrome extension for passive security analysis

Art Forms in Nature

Made this website because the Google results answering this question were wrong

We found cryptography bugs in the elliptic library using Wycheproof

Cloudflare is down and causing outages at X, OpenAI

Mpgr: Central task overlay for multi-repo workspaces

Cloudflare Global Network experiencing issues

Comments

Multiple Digital Ocean services down

Kerala becomes first state in India to eradicate extreme poverty

It's not surprising that 95% of AI enterprise projects fail

Towards interplanetary QUIC traffic with Rust Quinn

Google Gemini 3 Pro Model Card [pdf]

The Ochre Origins of Art

Show HN: Filtered GitHub Trends

SplitFlow: Flow Decomposition for Inversion-Free Text-to-Image Editing

GoSign Desktop RCE flaws affecting users in Italy

Sustaining Open Source: The Next 25 Years Depend on What We Do Together Now

Ask HN: Why is there black line on top? I guess not because of Cloudflare outage

Mathematics and Computation (2019) [pdf]

Apakah WhatsApp Bank Permata 24 Jam

Avi Wigderson – P vs. NP [video]

Do We Need to Slash the Debt?

The nature of the Theory of Computation (2018) [pdf]

Ask HN: Does insurance cover payouts during a global outage?

A DST primer for unit test maxxers

Show HN: Tool to check risks of your startup

Technoblogy – USB-C Power Delivery Dongle

Open Source in Focus: Projects We're Proud to Support

Show HN: An open source static site that parses and visualizes log files locally

MaggieLab – Simple, non-destructive, online image editor

Open Source Power

Show HN: SecuriScan – Open-source Chrome extension for passive security analysis

Art Forms in Nature

Made this website because the Google results answering this question were wrong

We found cryptography bugs in the elliptic library using Wycheproof

Cloudflare is down and causing outages at X, OpenAI

Mpgr: Central task overlay for multi-repo workspaces