so… they built a 4PB cluster, 14k cores, 50 GPUs, and it costs them less than my prod Kubernetes cluster on GCP. feels bad.
Growther0506•1h ago
dont worry, their real cost was their sanity
td17191719•1h ago
and what about the team of 1000 engineers you need to do the work of AWS support?
alexandra494•1h ago
This is maybe the first “ditch the cloud” post that isn’t totally ideological. Makes sense for their scale + workload. I wish more companies ran the numbers instead of just accepting AWS bills as fate.
idk123456•1h ago
the part about “predictable heavy workloads” should be bolded in neon. 90% of startups THINK they have predictable workloads. they do not. these guys actually do.
stefan420•1h ago
i would argue 90% of startups think they are going to need autoscaling, invest in that, and then never reach the scale up point, while having spent the investment money already.
besides the point of the article, tho. even if at the end of the day the numbers don't perfectly add up, i don't believe they are WAY OFF. we are paying for cables and HDDs man. "ancient" tech.
DaveEliot•1h ago
idk man, running your own hardware sounds awesome until you have to replace a dead RAID controller at 3am. article doesn’t really mention that part.
mihaivinaga•1h ago
Well, we do have some servers that we have in the office(the gpus servers, those have been a pain in the... mostly because intel cpus suck). But other then these the entire hardware part is managed by hetzner (thank you hetzner for being awesome)
iancu•1h ago
good story, but "works for us, so it should work for everyone". most startups can't touch this.
Growther0506•1h ago
the irony is cloud providers would love to sell a customer like this giant blocks of reserved compute. wonder if they actually negotiated or just said “lol too expensive” and bailed.
radu26•1h ago
not sure i buy the “$2M/mo on cloud” number. did they actually run it or just plug public on-demand pricing into a spreadsheet? nobody pays sticker price at that scale.
mihaivinaga•1h ago
That's a fair point, the scale discounts certainly matter. That said:
We did in fact run our prototype on AWS, and early on they flagged our account because our VMs were under heavy CPU load (we were told we'd need to cut below ~50 % or risk suspension).
When you switch to dedicated hardware, you're no longer paying for "noisy neighbours" or hypervisor over-provisioning, so the effective per-unit cost often cleaner fully justifies the shift.
alextofan•1h ago
Really appreciate how this isn't just another "cloud vs bare metal" holy war.The "own the core, rent the periphery" approach is exactly the kind of nuanced thinking we need more of in these discussions.
That said,important caveat for anyone reading this: Veridion is processing petabytes of data with predictable workloads. If you're still figuring out product-market fit or your usage patterns are all over the place, premature infrastructure optimization will absolutely kill you.The cloud's "boring reliability" has real value when you need to focus on building the actual product.
The 30x multiplier is real, but it only kicks in at serious scale with stable, known workloads. Most startups aren't there yet, and that's fine.
as3413•1h ago
ngl the cloud evangelists aren’t gonna like this one. this is basically “we did the math and cloud wasn’t worth it
martis2018•1h ago
not saying they’re wrong but $2m → $60k is like… a 97% cost drop. that is either cloud pricing insanity or some very optimistic bare-metal accounting.
cs0min•48m ago
can’t tell if they’re geniuses or just suffering a massive case of “we built a data center so now everything looks like a hardware nail."
and1-1•1h ago
Growther0506•1h ago