It's terrible. Gemini 2.5 Pro is great, but the AI overviews must be using a smaller model. I hate it when I look up something niche and it smugly tells me that I must be mistaken because there is no such thing. Also it gives annoyingly family-friendly responses to questions that it would be better off not responding to. The other day I was trying to find a Sopranos quote about two kinds of businesses being recession-proof, one of which being "certain aspects of entertainment" (i.e. prostitution) and it was telling me the certain aspects were filmmaking and music because they make people happy.

LLMs can't do math.

I recently used Gemini and Google search (with overview) to confirm whether a snack i bought from japan has expired. Used gemini to take a picture of the label written in japanese

One item said 25/7/25 the other one said 25/7/24 as you can imagine I was sure the first one was safe but the second one was confusing.

It told me that it's safe to eat because japanese date format is Year / Month / Date.

I looked up japanese date format in google (with overview) just to confirm. I guess we'll find out. Will report back soon.

I think I’d call these examples “predictable” failures instead of “odd”.

They are awful often for me. Examples - recommending installation of packages and software that doesn't exist, or settings changes that don't exist I In applications, etc. They fill the page but it's sadly noise so it cheapens the whole experience when I would have just preferred a link to a page from a person that knows what the hell they are talking about.

Why would you use an LLM for this? A simple spreadsheet can do this sort of calculation easily and deterministically.

Also, the assumption of '3% interest' is wrong. There are records of stretches achieving 15% returns for several years and reaching 23% in 2007, for example.

https://www.bloomberg.com/news/articles/2005-01-11/harvard-l...

https://www.wsj.com/articles/SB118771455093604172

This was 2 minutes of old school search, no LLM needed.

I saw a report via Simon Willison that if you make up a phrase and add "meaning" to the end of your Google search, it'll invent a meaning for it.

His example was "A swan won't prevent a hurricane meaning"

https://simonwillison.net/2025/Apr/23/meaning-slop/

It seems expectedly error prone.

Aside from the general limitations of this technology, Google needs this to be quite cheap if it runs for every request.

There is not a lot of revenue for a single search, and right now the AI results are actually pushing the links people are paying Google to display further down the page.

We're all sadly gullible.

We're all in IT. We know what an LLM is. But still we're fooled!?

I think it is because it is using a "mini" model with the search results as a RAG source so they can afford to use it on every single query. Thus it doesn't know very much and doesn't have much context to work with.

LLMs are all bad at math. But there are worse ways Google fails.

Like people asked "does Lululemon use <name of some chinese company> to make its products" and Google says "yes", with no source except one tiktok video that falsely claims it to boost sales in face of tariffs. (Ignoring that it's not in the actual supplier list published by lululemon on their site)

Which means basically people would see that tiktok, go to fact check on google if it's true, and google overview will say "yes" (+ paragraphs of text that no one reads) citing that tiktok.

Vicious circle of LLM factchecking. Google used to be immune to it until it started to shove chatbot output to people's faces.

I recently searched for a person and it concatenated the lives of several different people with the same name together like “X is a senator… He is also a professional baseball player…”

There was this Reddit post yesterday where it completely makes up being able to plant flowers in Elden Ring https://www.reddit.com/r/Eldenring/comments/1k6hupy/thanks_g...

The AI overview is worse than useless. It either hallucinates things or it treats shitposts as equally valid information-wise as anything else.

Our robot overlords are _terrible_ at anything financial. I'm on a financial forum where, lately, people are _constantly_ posting stuff that they got from The Oracle and asking what they're doing wrong because they don't understand the result, and the answer is inevitably that ChatGPT or whatever fed them plausible-looking rubbish (this is the _real_ AI safety problem; laypeople tend to take the output as infallible, even though it's usually rubbish.)

Though also as a sidenote Harvard's endowment probably wasn't put in a bank account with a flat 3% interest rate for a few hundred years...

It's not "AI" - its an LLM that is doing the equivalent of a college freshman padding out a paper for length. Confident, verbose, polished, but ultimately based on little hard reasoning - aka bullshitting. When it's wrong, and if you notice it and call it out, it will happily apologize and "correct" itself with the same well-written prose while making another mistake (or even the same exact one). LLMs certainly have utility, but it's more as generating inputs to some verifying processes rather than as a standalone oracle that competently answers questions.

It's not the AI doing math wrong as a lot of people are commenting. It's the way it's parsing your sentence. When it reads 'the present value of $100' it thinks that today's value of the investment is $100, and it needs to determine what the investment was worth 386 years ago (assuming a 3% interest rate).

Just invent a 'common' saying and add 'explanation' at the end.

Totally resonate with this. It feels less like a helpful overview and more like a confidently wrong pop-up I have to dismiss or fact-check before I can get to the actual search results (the links below).

I saw one example where someone asked about the fastest way to boil water, and the AI overview confidently stated that adding salt lowers the boiling point significantly, making it boil faster. It sounds vaguely scientific but gets a fundamental concept completely backward! That's the kind of error that's more worrying than just bad math – it confidently misrepresents basic, easily verifiable science.

It's a strange feeling having to approach Google search results with a layer of skepticism now, which used to be the gold standard for getting quickly pointed to reliable info. The AI Overview feels like a glossy, sometimes misleading, advertisement for the links I actually wanted in the first place.

It's hilariously bad. For one thing, it seems to try to find an 'explanation' for anything you type in, no matter how ridiculous that might be or if there's even any info on it online at all. It's become a common meme online to just come up with random fake idioms, throw them into Google and see the nonsense it comes up with in its desperate attempt to make sense of the non phrase.

And even if you do ask a legitimate question, you have to then hope the system knows what you actually mean rather than taking every word in your question literally and returning a complete non answer. So you might ask "was [actor name] in Chicago (the movie)?", only for Google to say "no, [actor name] doesn't live in Chicago".

Add the dangerous misinformation, the extremist answers sometimes generated and its attempts to make up sources if I can't find any, and well, it's basically useless for just about everything.

Call me old fashioned but I’ve been searching and using Google list results instead of the AI Overview.

Btw Google, you’re welcome I’m clicking links and making you money. For my needs, you’re a great search engine.

As far as I can tell, these aren’t made by asking a competent model to answer the “question” — based on what they seem to do, it seems to me like they take a model (a “mini” type of model?), pipe in the contents of the first 5 or 10 results from the slopfest that are Google’s current search results, and tell it to summarize THAT.

This is why it tells you to eat rocks. It is a very narrow sample of webpages and suffers from not even contextualizing each page it is reading to wonder if it’s a troll, satire, propaganda, fiction, or fact.

I have taken to ignoring them completely. I’d rather ask ChatGPT directly than trust these - and often I do just that. It’s much more accurate.

What’s frustrating is that the real estate these occupy was until a few years ago where they’d put text extracts quoted directly from a short-ish list of reputable sites. Same purpose, different content. While it was arguably a bit abusive of the sites to extract and display their contents there, the information used to be pretty reliable as a result.