AI datacenters are like nuclear power plants. Each user only consumes a fraction of its (compute) power but no single user can afford to build one themselves. There is no "tiny" nuclear power plant.
While local models do work, frontier models (like GLM 5.2) require huge amounts of memory which translates to high upfront costs that in the end demand high utilization to increase the ROI - if you can even afford such machines. This engineering effort is also adding to the cost of the hardware.
Because of these factors, models like GLM 5.2 will still be served from data centers for a while (just not the ones of OpenAI, Anthropic, Google etc.) until the hardware becomes so cheap that you can afford to keep it not fully utilized.
sourcediver•3h ago
While local models do work, frontier models (like GLM 5.2) require huge amounts of memory which translates to high upfront costs that in the end demand high utilization to increase the ROI - if you can even afford such machines. This engineering effort is also adding to the cost of the hardware. Because of these factors, models like GLM 5.2 will still be served from data centers for a while (just not the ones of OpenAI, Anthropic, Google etc.) until the hardware becomes so cheap that you can afford to keep it not fully utilized.
onion2k•2h ago
On the contrary: https://en.wikipedia.org/wiki/Small_modular_reactor
It's early days but plenty of companies are trying to commercialize designs for them, including Rolls Royce.