The relevant question is how easy is to setup 56x B200s in a cluster, are we talking about a adhoc infra team job or clicking a button on AWS?
So they've invested around $2.5M-3M into hardware + monthly operating costs (electricity, updates, cooling). I wonder how much did they spend on tokens to decide to make such a switch. I mean if they used to spend $500k-1M per month just on tokens then it would pay off in a year or so, assuming their infra will be stable and efficient.
We'll see if that's a trend for mid-big companies or just an experiment.
All that math is assuming that it's all self-hosted in their data center or so and not paying to a cloud provider, which will be probably more expensive.
The slice of the pie where frontier models are required is just gonna be smaller and smaller. And at point where AI wrote docs, tests, and all the easy and medium difficulty functions you are already saving a ton per employee
But the goal posts will move and I think frontier labs will find new areas that have a need for absolute frontier.
swyx•23m ago