frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

Open in hackernews

Webflow Down for >31 Hours

https://status.webflow.com
113•philip1209•13h ago

Comments

dangoodmanUT•12h ago
Hugs for their SREs sweating bullets rn
wavemode•12h ago
CEO's statement: https://www.reddit.com/r/webflow/comments/1mcmxco/from_webfl...
edoceo•12h ago
Interesting the phrase "I'm sorry" was in there. Almost feels like someone in the Big Chair taking a bit of responsibility. Cheers to that.
progbits•12h ago
> 99.99%+ uptime is the standard we need to meet, and lately, we haven’t.

Four nines is not what I would be citing at this point. (That's less than an hour per year, so they burned that for next three decades)

Maybe aim for 99% first.

Otherwise a pretty honest and solid response, kudos for that!

Spivak•12h ago
I strive for one 9, thank you. No need to overcomplicate. We use Lambda on top of Glacier.
jeeyoungk•12h ago
why go for 9's when you can go for 8s? you can aim for 88.8888888!
hnlmorg•12h ago
Hit that and you also master time travel.
aspenmayer•5h ago
I know you were going for a BTTF reference, but a Primer (2004) reference would be a better fit for a VC forum.

https://en.wikipedia.org/wiki/Primer_(film)

hinkley•12h ago
There's an old rant I cannot find at the moment that argued that most systems that believe they are 5 9's are really more like 5 8's.
theideaofcoffee•12h ago
Lots get starry-eyed and aim for five nines right out of the gate where they should have been targeting nine fives and learning from that. Walk before you run.
zamadatix•12h ago
One could have nearly 3 such incidents per year and still have hit 99%.

I always strive for 7 9s myself, just not necessarily consecutive digits.

manquer•10h ago
It could be consecutive too and even start with a 9 and be all nines here you go : 9.9999999%
thih9•12h ago
> Change controls are tighter, and we’re investing in long-term performance improvements, especially in the CMS.

This reads as if overall performance was an afterthought and this doesn’t seem practical; it should be a business metric, it is important to the users after all.

Then again, it’s easy to comment like this in hindsight. We’ll see what happens long term.

stackskipton•12h ago
I mean, if customers don’t leave them over this, higher ups likely won’t care after dust settles.
newZWhoDis•11h ago
As a former webflow customer I can assure you performance was always an afterthought.
bravesoul2•7h ago
Decent update. Guess people are really waiting for a fix tho!
nusl•12h ago
My sympathy for those in the mud dealing with this. Never a fun place to be. Hope y'all figure it out and manage to de-stress :)
pton_xd•12h ago
Claude, here is the bug, fix it. This is the new log output, fix the error. Fix the bug. Try a different approach. Reimplement the tests you modified. The bug is still happening, fix it. Fix the error.

We're out of credits, create a new account. We've been API rate limited? When did that start happening? When are we going to get access again?

Good luck engineers of the future!

ed_mercer•12h ago
You forgot to add “think hard!” :)
esafak•12h ago
And a subtle threat: "... or else".
lgl•12h ago
Comment of the year 2025! Thanks for that :D
zvmaz•12h ago
How do you know?
troyvit•11h ago
More like "Good luck users of the future" that have to wade through failing infrastructure and tools that were vibe coded to begin with, rate limits notwithstanding.
bravesoul2•12h ago
Wow >31h I am surprised they couldnt rebuild their entire systems in parallel on new infra in that time. Can be hard if data loss is invokved tho (a guess). Would love to see the post mortem so we all can learn.
stackskipton•12h ago
I doubt it’s infra failure but software failure. Their bad design has caught up and they can’t toss more hardware for some reason. Most companies have this https://xkcd.com/2347/ in their stack and it’s fallen over.
sangeeth96•12h ago
Hugs to the ones dealing with this and the users of Webflow who invested in them for their clientele. Hoping they'll release a full postmortem once the sky clears up.
ChrisArchitect•12h ago
Incident link: https://status.webflow.com/incidents/0xg8xq3l0h0q
nojs•11h ago
Wow, that whole page does not inspire confidence. It’s 99% LLM slop.

What We’re Doing:

-We are making ongoing adjustments to our infrastructure to improve stability and ensure reliable scaling under elevated load

-Analyzing system patterns and optimizing backend processes where resource contention is highest

-Implementing protective measures to safeguard platform integrity

esseph•11h ago
Expect every thing you read from here on our to be "AI Slop".

It's not going to get better in any way.

stackskipton•12h ago
My SRE brain reading between the lines is they have been feature factory and tech debt finally caught up to them.

My guess is reason they been down so long is they don’t have good rollback so they attempting to fix forward with limited success.

esafak•11h ago
If so it is probably a good time to apply for an SRE position there unless they really do not get it!
qcnguy•1h ago
More likely that their core database hit some scaling limit and fell over. Their status page talks constantly about them working with their "upstream database provider" (presumably AWS) to find a fix.

My guess. They use AWS hosted Postgresql and autovacuuming fell permanently behind without them noticing, and can't keep up with organic growth, and they can't scale vertically because they already maxed that out before. So they have to do crash migrations of data off their core DB which is why it's taking so long.

xyst•12h ago
I have no clue of "webflow" purpose based on it's marketing/buzzword filled landing page, but seems it's just a "no code" abstraction on top of HTML/CSS?

yet another SaaS that really does not need to be online 24/7. It could have been a simple app where you could "no code" on local machine and async state with webflow servers.

dylan604•12h ago
if you have a web based SaaS, everyone gets the updates. if you have a "simple app", then you are dependent on all of the users being up to date which you just cannot guarantee. also, what is a "simple app" that does not care about differences among various OSes found in the wild? how large of a team do you need for each of those OSes to support as wide of a user base as a web only app?
esseph•11h ago
Cost of having a reliable product with some self-determination for the customer.
dylan604•11h ago
the customer can self-determine just fine using a web based SaaS no-code website builder. it's not like this is a different type of app. the thing is making a website that is more also hosted by the maker of the app. if you want to make a website to host on your own servers, then you are not the target audience of the web app.

you're like the person complaining that the hammer isn't very useful for driving in the screw. you need a different tool/app if you want to make a site you host yourself

douglee650•1h ago
It's painful to use, but lets non-technical clients edit copy and create content in a safe environment. There's a runtime CMS types creator and a WYSIWYG html editor with facility for code blocks from global to inline scopes. Also comes with batteries included deploy. It's basically a one or two levels higher Squarespace/Wix
mattbillenstein•12h ago
We're sorry https://www.youtube.com/watch?v=9u0EL_u4nvw

Edit, an outage of this length smells of bad systems architecture...

hinkley•12h ago
Prediction: Someone confidently broke something, then confidently 'fixed' it, with the consequence of breaking more things instead. And now they have either been pulled off of the cleanup work or they wish they had been.
betaby•12h ago
I'm more surprised that WordPress-like platforms are profitable businesses in 2025.
bogzz•12h ago
Why? Genuinely asking. Did you mean because there are free alternatives to self-host? I don't think that it would be so easy for someone in the market for a WYSIWYG blog builder to set everything up themselves.
betaby•11h ago
Exactly. Because of the abundance of the one-click deploy WordPress offerings from value providers like OVH / Hetzner I would think margins are very low for WYSIWYG site builders.
esseph•11h ago
Decent demand just awful margins.

And most non-tech (and many in tech) have never heard about OVH/Hetzner.

newZWhoDis•11h ago
We moved away from webflow because it was slow (got the nickname web-slow internally).

Plus, despite marketing begging for the WYSIWYG interface they actually weren't creative enough to generate new content at a pace that required it.

We massively increased conversion rates by going full native and having 1 Engineer churn out parts kits/kitbash LPs from said kits.

Scale for reference: ~$10M/month

bravesoul2•7h ago
Because imagine your local biz can either pay a designer 1k a year or DIY and pay godaddy 200 bucks. Or 30 bucks for Wordpress and 20 hours of fiddling and asking their cousin for help.

Its not great by our standards but I bet many of us drink the house wine not something more sophisticated, right :)

wewewedxfgdf•12h ago
Companies get very good at handling disasters - after the disaster has happened.
dylan604•12h ago
The problem is they get good in that specific disaster. They can only plug a hole in the dike after the hole exists, then they look at the hole and make a plug the exact shape of that hole. The next hole starts the process over for it specifically. Each time. There's no generic plug that can be used each time. So sure, the get very good at making specific plugs. They never get to the point of making a better dike that doesn't spring so many leaks.
wewewedxfgdf•12h ago
It is the job of the CTO to ensure the company has anticipated as many as possible such situations.

It's not a very interesting thing to do however.

dylan604•12h ago
okay. and? the CTO isn't the last word in anything. if they are overruled to keep releasing new features, acquiring new users/clients, sales forward dev cycles, then the whole thing has potential to collapse under the weight of itself.

It's actually the job of the CEO to keep all of the c-suite people doing jobs. Doesn't seem to stop the CEO salary explosions.

wewewedxfgdf•12h ago
I think we are agreed.

Companies, after a disaster, focus lots of effort on that particular disaster, leaving all the other potential disasters unplanned for.

If you work at Webflow, you can anticipate LOTS of work in disaster recovery in the next 12 months. This has magically become a high priority for the CEO, who previously wanted features more than disaster recovery planning.

They will wait to focus massive resources on their security until after they get hacked.

esseph•11h ago
You just described every company.

(And also why security is always a losing battle)

willejs•12h ago
Hugops to the people working on this for the last 31+ hours. Running incidents of this significance is hard, draining and requires a lot of effort, this going on for so long must be very difficult for all involved.
bravesoul2•7h ago
Hopefully they are rotating teams not people staying awake for a dangerous amount of time.
acedTrex•12h ago
An outage of this magnitude is almost ALWAYS the direct and immediate fault of senior leaderships priorities and focus. Pushing too hard in some areas, not listening to engineers on needed maintenance tasks etc.
AsmodiusVI•10h ago
And engineers never are the cause of mistakes? There can't possibly be any data to back up that major outages are more often caused by leadership. I've been in SIEs simply because someone pushed a network outage to a switch network. Statements like these only go to show how much we have to learn, humble ourselves, and stop blaming others all the time.
AlotOfReading•10h ago
Leadership can include engineers responsible for technical priorities. If you're down for that long though, it's usually an organizational fuck-up because the priorities didn't include identifying and mitigating systemic failure modes. The proximate cause isn't all that important and the people who set organizational priorities are by-and-large not engineers.
acedTrex•9h ago
PROLONGED outages are a failure point that more often than not, require organizational dysfunction to happen.
bravesoul2•7h ago
Think of airplane safety. I think it is similar. A good culture can make sure $root-cause is more likely detected, tested, isolated, monitored, easy to roll back and so on.
plutaniano•11h ago
Will the company survive long enough to produce a postmortem?
ActionHank•11h ago
So now they’re Webno?
chupchap•11h ago
Bring back Failwhale
Marciplan•11h ago
y’all relax they are vibe coding the fix right now

China Prepares to Unseat US in Fight for $4.8T AI Market

https://www.bloomberg.com/news/articles/2025-07-30/china-prepares-to-unseat-us-in-fight-for-4-8-trillion-ai-market
1•artninja1988•1m ago•0 comments

Apple Promotes Samsung's Galaxy Z Flip7 by Mistake

https://www.macrumors.com/2025/07/30/apple-promotes-samsungs-galaxy-z-flip7-by-mistake/
1•tosh•2m ago•0 comments

Build Custom ActiveStorage Analyzers for Ruby on Rails

https://blog.appsignal.com/2025/07/30/build-custom-activestorage-analyzers-for-ruby-on-rails.html
1•amalinovic•3m ago•0 comments

LLM assisted language learning using Babbelaar

https://github.com/sjoerdvanderhoorn/babbelaar
1•sjoerdvdhoorn•4m ago•1 comments

Jokr.bar

https://jokr.bar/
1•sberkay•10m ago•0 comments

From XML to JSON to CBOR

https://cborbook.com/introduction/from_xml_to_json_to_cbor.html
1•GarethX•11m ago•0 comments

Why I recommend native Prometheus instrumentation over OpenTelemetry

https://promlabs.com/blog/2025/07/17/why-i-recommend-native-prometheus-instrumentation-over-opentelemetry/
1•gpi•13m ago•0 comments

QuickSheet – Instantly create and edit spreadsheets from your macOS menu bar

https://apps.apple.com/us/app/quicksheet-light-spreadsheet/id6748457063?mt=12
1•GnomeKev•14m ago•1 comments

Auto-Color Backdoor Targets U.S. Chemical Firm via CVE-2025-31324

https://sensorstechforum.com/auto-color-backdoor-cve-2025-31324/
1•cyberm1nd•18m ago•0 comments

The Untrustworthy and Rewritten

https://medium.com/@illuminati-ganga/the-untrustworthy-and-rewritten-707fdecb8cfc
1•bryanrasmussen•19m ago•0 comments

Ask HN: Tool that generates B2B sales briefs from URLs, feedback?

1•samehsbs•21m ago•2 comments

Built a new app for real treasure hunt in your city, the prize is credit card

1•DropCatch•22m ago•0 comments

Show HN: Go-DDD: Production-ready DDD patterns and clean architecture in Go

https://github.com/sklinkert/go-ddd
1•sklinkert•26m ago•0 comments

Ofcom Pressured Rumble and Reddit to Enforce UK Laws Beyond Borders

https://reclaimthenet.org/ofcom-pressures-rumble-reddit-to-enforce-uk-online-speech-laws
1•intunderflow•27m ago•0 comments

China's massive ADAS test: 36 cars, 15 hazard scenarios, 216 crashes

https://carnewschina.com/2025/07/24/chinas-massive-adas-test-36-cars-15-hazard-scenarios-216-crashes/
1•xrayarx•29m ago•2 comments

Avoid IaaS Lock-In with a SAML Proxy

https://mikehadlow.com/posts/2025-07-17-avoid-identity-vendor-lock-in/
1•mikehadlow•32m ago•0 comments

Managing Alpine Linux with Sparrow automation tool

https://asciinema.org/a/730670
1•melezhik•33m ago•0 comments

An LV Festival Promised Ways to Cheat Death. Two Left Fighting for Their Lives

https://www.propublica.org/article/peptide-injections-raadfest-rfk-jr
2•dbcooper•34m ago•0 comments

The Search for the Next Perfect Peanut

https://www.nytimes.com/2025/07/24/dining/peanut.html
2•bookofjoe•35m ago•1 comments

Less rain, more wheat: How Australian farmers defied climate doom

https://www.reuters.com/investigations/less-rain-more-wheat-how-australian-farmers-defied-climate-doom-2025-07-29/
3•tiarafawn•35m ago•0 comments

Phoenix LiveView v1.1.0

https://hexdocs.pm/phoenix_live_view/changelog.html#moving-from-floki-to-lazyhtml
1•Bogdanp•36m ago•0 comments

The Reanimated Renaissance, C-Speed Compression, and JSON-Loving LLMs

https://thereactnativerewind.com/issues-blog-post/the-reanimated-renaissance-c-speed-compression-and-json-loving-llms
1•lukebrandon•39m ago•1 comments

A major AI training data set contains millions of examples of personal data

https://www.technologyreview.com/2025/07/18/1120466/a-major-ai-training-data-set-contains-millions-of-examples-of-personal-data/
15•pera•43m ago•7 comments

Klarna Secures New License from UK's Financial Conduct Authority

https://www.bloomberg.com/news/articles/2025-07-30/klarna-secures-new-license-from-uk-s-financial-conduct-authority
1•thm•49m ago•0 comments

JPMorgan spooks fintechs with plans to charge for access to customer data

https://www.ft.com/content/85ccc517-4a34-4221-aec7-4d8a72a21f35
2•thm•49m ago•0 comments

How AI Is Replacing Manual Code Reviews–and Boosting Innovation

https://www.techolution.com/blog/ai-code-review-automation-uncovers-hidden-technical-debt-risks/
2•WoodenDist2857•50m ago•0 comments

A Quantum Gravimeter for GPS Backup

https://spectrum.ieee.org/quantum-gravity-sensor
1•pseudolus•54m ago•0 comments

Show HN: Free Tool to Convert 3-Hour YouTube Podcasts to MP3

https://www.aithumbnail.so/tools/youtube-to-mp3-converter
1•sachou•56m ago•0 comments

Bot vs. Bot: Will the Internet Soon Be a Place Without Humans? (2018)

https://singularityhub.com/2018/07/07/bot-vs-bot-will-the-internet-soon-be-a-place-without-humans/
1•hoyd•56m ago•1 comments

Bunup – blazing-fast build tool for TypeScript libraries

https://bunup.dev/
1•danfry•57m ago•0 comments