Ҩ.¬_¬.Ҩ
A miles per gallon number for a car doesn't count the diesel that went into the equipment to mine the ore to make the steel for the chassis, etc.
I was less surprised that inference dominates training after I read that chatgpt is serving billions of requests per day.
And unlike the human who spent multiple hours writing that article, an LLM would have linked to the original study: https://services.google.com/fh/files/misc/measuring_the_envi...
[ETA] Extending on these numbers a bit, a mean human uses 1.25KW of power (Kardashev Level .7 / 8 Gigahumans) and the mean American uses ~8KW of power according to https://en.wikipedia.org/wiki/List_of_countries_by_energy_co.... So if we align AIs to be eco-friendly, they will definitely murder all humans for the sake of the planet /s
There is a non-trivial chance that the LLM would've added a link to _something_, but links/references seem like a very common thing to hallucinate, no?
The used results can then have their link either added to the end result separately, guaranteeing it is correct, or added to the prompt and "telling the LLM to include it", which retains a risk of hallucination, yes.
Common to both of these is the failure mode that the LLM can still hallucinate whilst "summarizing" the results, meaning you still have no guarantee that the claims made actually show up in the results.
Hardly better, as soon those "search engine results" would be AI slop themselves, including actual published papers (phoned-in by using AI, and "peer reviewed" by using AI from indifferent reviewers)
Would the LLM-based tool be able to determine that the top results are just SEO-spam sites and move lower in the list, or just accept the spam results as gospel?
And then on the rare occasion they do link to a chat, their prompt is something like:
"Tell about a person of history who was important for their work inthe time of their existence and give quotes of what they said that made them important when they were speaking include notes and other stories about them and give details about their life who they married and their kids and who their parents were and other big things they did do in their lives"
Instead of downvotes, please prove me wrong.
.24 Watt-hours is 864 Watts for one second, so a 100W human takes ~9 seconds for that output.
Also, since I live a first-world life style which consumes multiple KW of power, I've probably consumed multiple orders of magnitude energy more than an LLM on this topic.
Or made up a fake citation, complete with fake or unrelated author names, on the spot
Edit - just used Pro, gave me a direct source. Who knows...
Mmmh, that would have been my take as well up to around end of Q1 2025.
Theses days, the flagship LLM's have reduced hallucination by quite a bit, and are also way better at citing sources (you sometimes have to nudge them).
ChatGPT 5 has been very decent on that particular axis.
This is why journalists are nearly universally hostile towards AI.
I love good journalism because it's adhd crack; in-depth spilling the tea but you have to really dig deep to find it nowadays.
Not to mention that the energy should also include all the extra energy spent on making converting energy into a form that is usable by humans (ie. food). There is probably at least an order of magnitude.
Unless your point is that we can kill a bunch of humans to save energy...?
So it's not just about "the one query you ask ChatGPT about what you should write your mum to say you're not coming for Thanksgiving"
It's rather that an AI query is 0.24Wh, but that we are now using thousands of those per users per days, and that we globalize it at the scale of the planet, so 7 billion users... and this becomes huge
The numbers are cute but we can't actually do anything with them without those details. At least an average could be multiplied by the # of queries to get the total usage.
I'm honestly surprised that they're so similar. I've thought of LLM queries as being far more energy-intense than "just" a Google search, but maybe the takeaway is that ordinary Google searching is also quite energy-intense.
If I as a user just wanted an answer to a dumb question like, say, the meaning of some genZ slang, it seems about an order of magnitude to ask a small LLM running on my phone than to make a google search.
(Check my math: assuming the A16 CPU draws 5 watts peak for 20sec running Gemma or whatever on my iPhone, that’s 0.03Wh to answer a simple query, which is 10x cheaper)
Are training costs (esp. from failed runs) amortized in these estimates?
1: https://googleblog.blogspot.com/2009/01/powering-google-sear...
A related takeaway should be that machine inference is pervasive and has been for years, and that defining "AI" to mean just chatbots is to ignore most of the iceberg.
Not just "one training run," but the cost of a thousand AI engineers starting failing runs to get to that one deployed model.
1: Link to Google's tech report: https://services.google.com/fh/files/misc/measuring_the_envi... "We leave the measurement of AI model training to future work."
From 2022, so possibly out of date: "ML training and inference are only 10%–15% of Google’s total energy use for each of the last three years, each year split ⅗ for inference and ⅖ for training." That's probably close enough to estimate 50/50, or the full energy cost to deliver an AI result is double the inference energy.
https://research.google/blog/good-news-about-the-carbon-foot...
Various ML "learn-to-rank" tooling was in use at Google for a while, but incorporating document embedding vectors w/ ANN search into the ranking function probably happened over the course of 2018-2021 [1], I think. Generative AI only started appearing in ordinary search results in 2024.
1: https://cloud.google.com/blog/topics/developers-practitioner...
Inverted indices were not used as they worked poorly for “an ordered list of words” (as opposed to a bag of words).
And this doesn’t even start to address the ranking part.
> In total, the median prompt—one that falls in the middle of the range of energy demand—consumes 0.24 watt-hours of electricity
If they're running on, say, two RTX 6000s for a total draw of ~600 watts, that would be a response time of 1.44 seconds. So obviously the median prompt doesn't go to some high-end thinking model users have to pay for.
It's a very low number; for comparison, an electric vehicle might consume 82kWh to travel 363 miles. So that 0.24 watt-hours of energy is equivalent to driving 5.6 feet (1.7 meters) in such an EV.
When I hear reports that AI power demand is overloading electricity infrastructure, it always makes me think: Even before the AI boom, shouldn't we have a bunch of extra capacity under construction, ready for EV driving, induction stoves and heat-pump heating?
[1] https://cloud.google.com/blog/products/infrastructure/measur...
When it comes to the EV, the answer is simple: the EV takeover "by 2030" was 100% wishful thinking - the capacity is nowhere near there, starting from scaling the battery production, never mind the charge capacity.
Existence of “2030 deadline” was/ is significant factor by itself. (Current sate would be less electrified without that arbitrary and over optimistic fantasy deadline)
We'll have the battery capacity and charge capacity to allow 100% of cars sold in 2030 to be EV's. We only need 2 capacity doublings for batteries, and currently doublings happen every ~18 months. Charge capacity is even easier, we just need to increase electricity production by 1-2% per year for a couple decades to support the transition to EV's.
China however is continuously providing double the energy they currently require, only to notice that every two years or so it actually did end up getting used.
It feels like dog-whistle tactics. "Aren't the technology companies bad for the environment!" "What about the water usage?" "What about the electricity?"
For me the peak of this is complaining about water consumption at the Dalles datacentre [0]. The buildings are next to the Colombia river and a few miles away from the Dalles Dam [1] which generates an average of 700MW. The river water should be used for cooling, taking out some of the water, warming it up by a few degrees and returning it to the river; one might argue that this is simply returning the heat to the river that would have come from the water flowing downhill.
[0] https://www.oregonlive.com/silicon-forest/2022/12/googles-wa...
Where do you think the evaporated water goes?
Drinking water, spraying it on crops, using it to clean a car, or using it to flush a toilet all end up with the water evaporating, or making its way to the ocean and evaporating from there.
Ultimately, if a river provides a certain number of acre-feet of fresh water, evaporating it to cool a data centre uses it just as much as to evaporating it to grow alfalfa in a desert, except perhaps more usefully.
We do get new fresh water at a reasonable pace thanks to rain - but in many parts of the world we are using it faster than that, and not just depleting the stored volume of fresh water but destroying the storage "containers" themselves.
I think you're oversimplifying the "just use rivers" idea. Most data centers (80% for Google) require potable water for cooling, and it can't come straight from a river. Plus, using potable water in cooling adds mineral deposits to the water and will require treatment to be consumable again.
You're not accounting for batches for the optimal gpu utilization, maybe it can takes 30 seconds but it completed 30 requests.
> People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes. It also uses about 0.000085 gallons of water; roughly one fifteenth of a teaspoon.
That's.... a lot cheaper than I would have guessed. Obviously, the data centers cost quite a bit to build. But when you think of $20/mo for a typical subscription. That's not bad?
Did I do that right?
The fundamental flaw in AI energy/water doomerism has always been that energy costs money, water costs money, real estate costs money, but AI is being given away for free. There is obviously something wrong with the suggestion that AI is using all the energy and water.
That's equivalent to doing less than two miles driving(CO2), one toilet flush (water) and about three dryer loads of laundry.
We can always keep adding new stuff and say each time "oh but it's small"... sure, but if we keep adding more, altogether it becomes huge
Yeah, I was more interested in knowing the total amount. A "median" prompt without the information on the total number of prompts is kind of meaningless...
Eh, I somewhat disagree with this. The US energy grid has had almost no extra capacity for a long time. A lot of this was due to efficiency (not a bad thing) and little industrial growth (not a great thing) in the country. Data centers themselves, I don't think are the biggest cause of the issues, but the distribution grid. We've had tons of problems around distribution with new energy sources coming online, but issues distributing the power to where we need to.
We see things like private power plants, not because we can't generate power, but because we absolutely suck at distribution of power.
Still, there are a lot unswered questions here, and its up in the air precisely how this stuff will continue to integrate into services we already use, or habits we have yet to form at large. What does that scaling look like?
But by far the most troubling thing is the continued combination of flippancy, defensiveness, or silence we get from the AI peiple about even attempting to talk about this. If you are a True Believer, don't you want this to be something that is tackled head on, rather than tucked away? When this has come up before, I always end up seeing a bunch of guys who essentially leave the vibe of "well I am plenty above sea level, my AC is pumping just fine, and I just simply don't care because my productivity has doubled!"
Like isn't this community supposed to be excited about a future, eager to tackle problems? Or is there maybe some intrinsic solipsism to the impressive chatbots that ultimately renders this kind of attitude to its users? It feels like right when we were culturally about to age out of this particular form of obstinacy, we set ourselves up to create a whole new generation of "global warming is fake news" people. Which is a shame. If you're going to be like this, just go all in on accelerationism in all its pseudo-facist darkness, don't just borrow a script from baby boomers!
the median prompt [...] consumes 0.24 watt-hours of electricity
- one second of running a toaster, or
- 1/80th of a phone charge,
- lifting 100 pounds to a height of 6 feet,
- muzzle energy of a 9mm bullet,
- driving 6 feet with a Tesla.
There is a perception out there about GenAI and water that goes surprisingly deep. I was told we are will be living in a drought-stricken hellscape, and AI is to blame.
I'd like to know the equivalent energy consumption of a single TikTok video, but that is probably arguing the wrong thing. My bigger question is ... where do they think that water goes? Steam? The assumption is that it is gone forever, and I can't get over how people could just take that at face value.
And there isn't solid evidence that this was connected to the data center construction:
> Ben Sheidler, a spokesman for the Joint Development Authority, which manages the industrial park that Meta’s facilities occupy, said the cause of the water issues was unknown. The Joint Development Authority did not do a well water study before construction to determine any potential effects, but the timing of the problems could be a coincidence, he said.
> “I wouldn’t want to speculate that even the construction had something to do with it,” he said. “One thousand feet away is a pretty significant distance.”
Data centers don't just heat up the water and return it - they evaporate the water into the atmosphere (yes, I know, the H2O still exists, but it's in a far less usable form when it's gaseous atmospheric H2O)
Blog post: https://cloud.google.com/blog/products/infrastructure/measur...
Paper: https://services.google.com/fh/files/misc/measuring_the_envi...
Is it a metric for marketing to beat competitors with, like GPU speeds, etc.
"We're more efficient than those Global Warming bastards over at Amazon."
I assume they wouldn't publish them if it cast them in a bad light.
In this thread alone there are many comments multiplying the median to get some sort of totalt, but that's just not how medians work.
If I multiplied my median food spent per day with the number of days per month, I'd get a vastly lower number than what my banking app says.
The median is more useful to answer the question of how much energy a typical Gemini interaction uses.
FridayoLeary•3h ago
>The report also finds that the total energy used to field a Gemini query has fallen dramatically over time. The median Gemini prompt used 33 times more energy in May 2024 than it did in May 2025, according to Google.