Totally agreed the news like this makes me nervous too. For a high profile customer it feels like Google should give a clear explanation of what happened.

This is actually scary. If Google can suspend a company like Railway without warning, what chance does a smaller startup have? The lack of any human escalation path at Google Cloud has been a known problem for years. You'd think enterprise customers paying real money would at least get a phone call before getting shut down

Yup, I think so. Makes one think about how dependent we are on cloud infra for core pieces versus supporting pieces of the architecture. They've probably negotiated some kind of private settlement.

Would you want your vendors publicly disclosing potentially private reasons for an outage?

It depends on what you mean by ‘need’. If you mean they should for PR purposes, I probably agree.

If you are saying they should be required to by law, then no I disagree.

Google has given a public statement about this category of incident (to wit: cloud provider imperils customer's operations by way of automated decision deliberately designed to withhold recourse).

That statement is the last 15 or so years.

I think so. I'm a GCP user and I'm afraid of hosting workloads there now. I've heard too many nightmare stories, and I thought Google would be proper and thus not be infested with these kind of problems that cheaper providers are known for.

Maybe AWS is the only player in town now? I don't know. Google doesn't instill confidence with these incidents, same with those cases of insurmountable bills caused by simple mistakes where there should be a way for smaller customers to cap usage.

Railway can simply move to other service. We all know Google in unreliable, so why should google give public statement?

Thanks.

I really don't see how they can. The business and usage details of their clients are confidential. We have their word that ToS where violated, I don't really think they should say more. This needs to go to arbitration.

Being an advocate for GCP all these years, I can only say the earlier you get out of it, the safer it is for your business. All it takes is for their automated system to go haywire, and you can say bye bye to all your goodwill and customers. Go look at twitter how many customers are blaming railway. Founder had history getting screwed by GCP, yet still choose to depend on them.

You can't rely your business on GCP. Honestly, this is the most silly way to kill your own business.

For context, copied from my post 3 years ago.

March 10, 2023 | hide | past | favorite As a 4 years customer, our production severs have been suspended by Google Cloud because we didn't fill up some information on-time. Contacted support but they expect us to wait for 24-48 hours to get it resolved while all our servers are down. Anyone linked with someone powerful in google cloud can help?

======

- Running production on google cloud for 4 yrs with my startup. 100% legit SaaS business.

- Always pay bills on-time no issue. Good customer never open tickets, ask for help or what just quietly pay my bills each month.

- Our servers was abruptly suspended yesterday midnight and my whole business is now down for > 10hrs.

- We run a SaaS business that other ecommerce stores rely on and have hundreds of paying merchants.

- My customers have been grilling me and I don't feel gcloud's trust and safety team understand/care how urgent the issue is.

======

Why were our servers suspended? Because we didn't fill up information in time?

- See https://imgur.com/a/x0Y3RJl

- Apparently they dropped us an email 10 days back that I missed out

- Titled "Important Information Regarding Your Google Account" with no indication of suspension or what in title.

- Given the number of subprocessor "Important" emails they send it's too easy to miss out the email.

- 10 days gone by and our servers were abruptly shutdown with zero suspension notification or what.

- We've been paying $400-$700/mo for the past 4 yrs consistently and they shut us down because we didn't fill up some information?

When I tried to ask them to at least temporarily get our servers back while the verification is ongoing, I didn't get any answers.

Google Cloud have zero empathy for customers.

It's not like my account got suspended for fradulent issue or what. It's suspended because I didn't fill up some information on-time and they don't even allow me to temporarily reactivate my services or what. Especially when I had to wait for hours to get their team to verify my details before I can get my servers back.

You can't trust them with your business. Don't run any production stuffs with Google Cloud, ever.

The company I work for uses GCP and we preciously had intermittent CloudSQL connection errors for a few hours. We reached out and they resolved it after a day or so and said there was a minor incident but I don’t think it was ever publically reported.

This is exactly why people get nervous about platform risk