A thought occurs. GPUs have a limited lifespan. Gpus die after 1-3 years of use.[1] Just in time to train an LLM or two. The data centers themselves without the GPUS is like 40% of the cost from what I hear. In 3 years from now when the boom ends, they are going to be empty warehouses, with very good networking and cooling.
True, they burn through GPUs. However, I wonder what the actual curve looks like... what fraction of total GPU capacity is getting maxed out, to the 3 year "burned to crisp" threshold. Training is harsher than inference is harsher than speculative capacity-hoarding (because, competition).
Even after that, what does a "burned out" GPU look like. Is it a total bust, or is still usable at... say, 25% capacity for "consumer type applications"?
Thank you for that GPU lifespan explanation... taught me a thing or two today.
throwawayffffas•1mo ago
> Even after that, what does a "burned out" GPU look like. Is it a total bust, or is still usable at... say, 25% capacity for "consumer type applications"?
From what a hear it's a mix, of completely dead to degraded performance.
> Training is harsher than inference is harsher than speculative capacity-hoarding (because, competition).
I have heard over 70% quoted used for training, and like 5% for general purpose inference and the rest for code generation. But don't quote me on these numbers, I don't recall the sources. One has to assume that some capacity is also used for traditional high performance computing.
adityaathalye•1mo ago
Educated guesstimates are worth a lot. Thank you for the stats!
adityaathalye•1mo ago
Still, I do wonder about the GPU manufacturing capacity upstream of datacenters, even though it grows relatively slowly. I suppose NVIDIA's order book is booked solid a few years out. However, capacity that they add can't just be repurposed / retooled for other use cases.
What could substitute LLM demand, if the LLM/AI business contracts rapidly?
adityaathalye•1mo ago
Another factor... They built it, and we didn't come.
Groq investor sounds alarm on data centers (axios.com)
32 points by giuliomagnifico 2 hours ago | 21 comments
> Venture capitalist Alex Davis is "deeply concerned" that too many data centers are being built without guaranteed tenants, according to a letter being sent this morning to his investors.
> [snip]
> What he's saying: "The 'build it and they will come' strategy is a trap. If you are a hyperscaler, you will own your own data centers. We foresee a significant financing crisis in 2027–2028 for speculative landlords."
fuzzfactor•1mo ago
>Could a "GPUs too cheap to meter" phase—say, about a decade, up to 2040—remarkably speed up cycle times of traditional deterministic modeling / simulation type workloads.
Seems to me for it to really go wild the bottleneck to overcome would need to include GPU experts too cheap to meter also.
adityaathalye•1mo ago
The people assembling GPU boxen would need work. Big Datacenter will likely turn off the power and evict those human cost centers, as a first step toward asset liquidation. Which is why I hope that occurrence feeds forward into a glorious SME business boom. e.g. Nokia's mobile telephony self-own by the end of the aughts was both sad and great for Finland; hurting national pride at one end and fuelling their high tech startup scene, exactly due to losing experts to attrition and entrepreneurship.
throwawayffffas•1mo ago
[1]. https://ithy.com/article/data-center-gpu-lifespan-explained-...
adityaathalye•1mo ago
Even after that, what does a "burned out" GPU look like. Is it a total bust, or is still usable at... say, 25% capacity for "consumer type applications"?
Thank you for that GPU lifespan explanation... taught me a thing or two today.
throwawayffffas•1mo ago
From what a hear it's a mix, of completely dead to degraded performance.
> Training is harsher than inference is harsher than speculative capacity-hoarding (because, competition).
I have heard over 70% quoted used for training, and like 5% for general purpose inference and the rest for code generation. But don't quote me on these numbers, I don't recall the sources. One has to assume that some capacity is also used for traditional high performance computing.
adityaathalye•1mo ago
adityaathalye•1mo ago
What could substitute LLM demand, if the LLM/AI business contracts rapidly?
adityaathalye•1mo ago
Groq investor sounds alarm on data centers (axios.com)
https://news.ycombinator.com/item?id=46432791> Venture capitalist Alex Davis is "deeply concerned" that too many data centers are being built without guaranteed tenants, according to a letter being sent this morning to his investors.
> [snip]
> What he's saying: "The 'build it and they will come' strategy is a trap. If you are a hyperscaler, you will own your own data centers. We foresee a significant financing crisis in 2027–2028 for speculative landlords."