Had a small discussion about this on an OP on bsky. A somewhat interesting discussion over there.

https://bsky.app/profile/simonpcouch.com/post/3mcuf3eazzs2c

LLMs don't use much energy at all to run, they use it all at the beginning for training, which is happening constantly right now.

TLDR this is, intentionally or not, an industry puff piece that completely misunderstands the problem.

Also, even if everyone is effectively running a a dishwasher cycle every day, this is still a problem that we can't just ignore, that's still a massive increase in ecological impact.

At first glance this looks like a credible set of calculations to me. Here's the conclusion:

> So, if I wanted to analogize the energy usage of my use of coding agents, it’s something like running the dishwasher an extra time each day, keeping an extra refrigerator, or skipping one drive to the grocery store in favor of biking there.

That's for someone spending about $15-$20 in a day on Claude Code, estimated at the equivalent of 4,400 "typical queries" to an LLM.

As long as it's unaccounted for by users it's at best anexternality. I think it may demand regulation to force this cost to the surface.

electricity and cooling incur wider costs and consequences.

That is a pretty good article although the one factor not mentioned that we see that has a huge impact on energy is batch size but that would be hard to estimate with the data he has.

We've only launched to friends and family but I'll share this here since its relevant: we have a service which actually optimizes and measures the energy of your AI use: https://portal.neuralwatt.com if you want to check it out. We also have a tools repo we put together that shows some demonstrations of surfacing energy metadata in to your tools: https://github.com/neuralwatt/neuralwatt-tools/

Our underlying technology is really about OS level energy optimization and datacenter grid flexibility so if you are on the pay by KWHr plan you get additional value as we continue to roll new optimizations out.

DM me with your email and I'd be happy to add some additional credits to you.

I'm not sure I like this method of accounting for it. The critics of LLMs tend to conflate the costs of training LLMs with the cost of generation. But this makes the opposite error: it pretends that training isn't happening as a consequence of consumer demand. There are enormous resources poured into it on an ongoing basis, so it feels like it needs to be amortized on top of the per-token generation costs.

At some point, we might end up in a steady state where the models are as good as they can be and the training arms race is over, but we're not there yet.

I have a kids and a dishwasher (which with kids, runs quite often) but I’m not convinced I’m doing worse at energy consumption

Only tangentially related, but today I found a repo that appears to be developed using AI assistance, and the costs for running the agents are reported in the PRs. For example, 50 USD to remove some code: https://github.com/coder/mux/pull/1658