https://www.youtube.com/live/0Uu_VJeVVfo?si=PJGU-MomCQP1tyPk
There are savvy people who know when to say "don't tell me that information" because then it is never a lie, simply "I was not aware"
Have any systems tried prompting LLMs with a warning like "You don't intuitively or automatically know many facts about words, spelling, or the structure or context of text, when considered as text; for example, you don't intuitively or automatically know how words or other texts are spelled, how many letters they contain, or what the result of applying some code, mechanical transformation, or substitution to a word or text is. Your natural guesses about these subjects are likely to be wrong as a result of how your training doesn't necessarily let you infer correct answers about them. If the content or structure of a word or text, or the result of using a transformation, code, or the like on a text, is a subject of conversation, or you are going to make a claim about it, always use a tool to confirm your intuitions."?
As a human, if you give me a number on screen like 100000000, I can't be totally sure if that's 100 Million or 1 Billion without getting close and counting carefully. Should ought have my glasses. Mouse pointer helps some as an ersatz thousands-separator, but still.
Since we're giving them tools, especially for math, it makes way more sense to start giving them access to some of the finest tools ever. Make an MCP into Mathematica or Matlab and let the LLM write some math and have classical solvers actually deal with the results. Let the LLM write little bits of bash or python as its primary approach for dealing with these kinds of analytical questions.
It's like giving a kid a calculator...
I become mildly infuriated when computers show metrics (or any large number) without thousands separators.
Worse still, I often see systems that mix units, don’t right-align, and occasionally blend in a few numbers with decimals together with whole numbers! Then, update everything every second to make things extra spicy.
It is for those using the LLM to be aware of its capabilities, or not - be allowed to - use it. Like a child unaware that running their finger on a sharp knife blade will lead to a bad slice; you don't dull the blade to keep the child safe, but keep the child from the knife until they can understand and respect its capabilities.
If the creation is planned, you will have also thought of the handle; if it is a serendipity, you will have to plan the handle afterwards.
LLMs do not deliver (they miss important qualities related to intelligence); they are here now; so they must be superseded.
There is no excuse: they must be fixed urgently.
No, I would call (an easy interpretation of) that an implementation of unintelligence. Following patterns is what an hearsay machine does.
The architecture you describe at the "token prediction" level collides with an architecture in which ideas get related with better justifications than frequent co-occurrance. Given that the outputs will be similar in form, and that "dubious guessers" are now in place, we are now bound to hurry towards the "certified guessers".
That's also how the brain works, at least partially. Primary differences are it takes and processes (trains itself on) raw sensory data instead of character tokens, and it continually does so for every conscious moment from at least birth until death.
With the difference, which have us go back to the original point, that the human mind has a crucial property of going beyond "pattern-based" intuition and check mental items lucidly and consciously.
> and it continually does so
It also persistently evaluates consciously and "store" and "learn" (which must be noted because it is the second main thing that LLMs don't do, after the problem of going past intuition).
Capabilities that evolved over millennia. We don't even have a decent, universally-agreed upon definition for consciousness yet.
> "store" and "learn"
Actually there are tools for that. Again, the core LLM functionality is best left on its own, and augmented on the fly with various tools which can be easily specialized and upgraded independently of the model. Consider too that the brain itself has multiple sections dedicated to different kinds of processing, instead of anything just happening anywhere.
Means nothing. Now they are urgent.
> consciousness
I meant "conscious" as the wake opposed to deliriousness, as the ability "to be sure about what you have in front of you with the degree of accomplished clarity and substantially unproblematic boundaries of definition",
not as that quality that intrigues (and obsesses) less pragmatic intellectuals in directions e.g. at the "Closer to Truth" channel.
When I ask somebody, it has to be sure to a high degree. When implementing a mind, the property of "lucid conscious check" is fundamental.
> tools for that
The "consciously check, then store and learn" is structural in the proper human mental process - a high level functioning, not just a module; i.e. "it's what we do".
Which means, the basic LLM architecture is missing important features that we need if we want to implement a developed interlocutor. And we need and want that.
Something being urgent doesn't mean there's a known viable pathway to an ideal implementation. The brain has billions of neurons working in tandem, a scale we're nowhere near to replicating, last I checked. And there are signs pointing to scale of neural interactions as a key factor in intelligent capabilities.
> When I ask somebody, it has to be sure to a high degree.
You're looking in the wrong place if you need surety. LLMs aren't "sure" about anything, and will never be. We going in circles at this point, but if you need certitude in anything, add tools to the LLM to increase surety. For some reason you seem to be pushing for a less optimal solution to a problem that already has a decent one.
BTW seems you may be - unconsciously? - crossing a person with an LLM there.
> structural in the proper human mental process
The brain is very modularized. It has sections/lobes which specialize in core life support functions, seeing, hearing, movement, reasoning, memory, etc. That's why brain surgeons can reliably know what capabilities may be affected by their actions in a given part of the brain. And all those functions are tools in some way to something else, for example reasoning would be pretty limited without memory.
And even at a larger scope, as humans we still use tools to achieve greater surety from our mental processes. That's why we have calculators, watches, cameras, etc. And why some would type "blueberry" into a tool with proven spell checking or autocorrect capabilities and eyeball the letters for a couple seconds to confirm the number of "b"s in it. The brain as a whole is still pretty fallible with all its capacity and capabilities.
It means it must be researched into with high commitment. // LLMs are an emergency patch at best (I would not call them «decent» - they are inherently crude). This is why I insist that they must be overcome with urgency already because they are now here (if a community needed wits and a lunatic appears, wits become more needed). // And no, I am not «crossing»: but people do that, hence I am stating an urgency.
We do not need to simulate the brain, we only ("only") need to implement intelligence. That means the opposite of stating hearsay: it means checking every potential uttering and storing results (and also reinforcing the ways that had the system achieve sophisticated thoughts and conclusions).
It is not given that LLMs cannot be part of such system. They surely have a lot of provisional uttering to criticize.
It could very well be that the highest intelligent functionalities requires a closely brain-like substrate. We don't know yet, but we'll get there eventually. And it is very likely to be something emergent, not specifically programmed features as you seem to be insinuating with "... checking [...] and storing results..."
The implementation details are not clear, not the goals.
I never said that the feature has to be coded explicitly. I said it has to be there.
The engineering will be different.
For things where factuality is even just a little important, you need to treat these things like asking a toddler that got their hands on a thesaurus and an encyclopaedia (that's a few years out of date): go through everything it produces and fact check any statement it makes that you're not confident about already.
Unfortunately, people seem to be mistaking LLMs for search engines more and more (no doubt thanks to attempts from LLM companies to make people think exactly that) so this will only get worse in the future. For now we can still catch these models out with simple examples, but as AI fuckups grow sparser, more people will think these things tell the actual truth.
4 "u" because "b" felt like a bit of a cheat to count in that sentence.
We are not interested specifically in the inability to «know» about text: we are strongly interested in general in the ability to process ideas consciously, procedurally - and the inability to count suggests the general critical fault.
A quick test anyone can run and say, yup, that is a model XYZ derivative running under the hood.
Because, as you quite rightly point out, it is trivial to train the model not to have this behaviour. For me, that is when Occam kicks in.
I remember initially believing the explanation for the Strawberry problem, but one day I sat down and thought about it, and realized it made absolutely zero sense.
The explanation that Karpathy was popularizing was that it has to do with tokenization.
However, models are not conscious of tokens, and they certainly don't have any ability to count them without tool help.
Additionally, if it were a tokenization issue, we would expect to spot the issue everywhere.
So yeah, I'm thinking it's a model tag or insignia of some kind, similar to the fun logos you find when examining many silicon integrated circuits under a microscope.
AI> Thinking... .oO{ Well, let's see. The letter b is the second letter in the alphabet, so the answer must be two, right? That sounds right to me anyway. }
AI> Two. The answer is two.
Checks out!
The letter “b” appears 3 times in the word blueberry: • 1st b – at the beginning: Blueberry • 2nd b – in the middle: blueberry • 3rd b – again in the middle: blueberry (wait… no — actually…)
Let’s break it down carefully:
blueberry → b, l, u, e, b, e, r, r, y
Letters: 1. b 2. l 3. u 4. e 5. b 6. e 7. r 8. r 9. y
So the letter b appears twice.
Final answer: 2 times.
But my browser has gpt-5 which says 3: https://files.catbox.moe/63qkce.jpg
Claude spells it out letter by letter: https://files.catbox.moe/f1irfx.jpg
So I thought GPT-5 Thinking might get it right, and it does: https://files.catbox.moe/xlchnr.jpg
It refuses to show the thinking process for this question though, so its unclear if it even used the reasoning model or fell back on a non reasoning one.
> While GPT‑5 in ChatGPT is a system of reasoning, non-reasoning, and router models, GPT‑5 in the API platform is the reasoning model that powers maximum performance in ChatGPT. Notably, GPT‑5 with minimal reasoning is a different model than the non-reasoning model in ChatGPT, and is better tuned for developers. The non-reasoning model used in ChatGPT is available as gpt-5-chat-latest.
Then I simply asked it to count all unique letters in the word. GPT 5 still got it completely correct without thinking.
Lastly I asked how many r(or b) is in the word. This one for some reason switched to GPT 5 thinking with few seconds of reasoning. It out put the correct number.
I guess starting the conversation by painstakingly walking it over to the correct answer helps it out. Idk it's a silly test
See
How many 538 do you see in 423, 4144, 9890?
It's really disingenuous for the industry to call warming tokens for output, "reasoning," as if some autocomplete before more autocomplete is all we needed to solve the issue of consciousness.
Edit: Letter frequency apparently has just become another scripted output, like doing arithmetic. LLMs don't have the ability to do this sort of work inherently, so they're trained to offload the task.
Edit: This comment appears to be wildly upvoted and downvoted. If you have anything to add besides reactionary voting, please contribute to the discussion.
Of course then you ask her to write it and of course things get fixed. But strange.
That is to say, you can obtain the same process by talking to "non-reasoning" models.
There will be a series of analytical articles in the mainstream press, the tech industry will write it off as a known problem with tokenisation that they can't fix because nobody really writes code anymore.
The LLM megacorp will just add a disclaimer: the software should not be used in legal actions concerning fruit companies and they disclaim all losses.
Mechanistic research at the leading labs has shown that LLMs actually do math in token form up to certain scale of difficulty.
> This is a real-time, unedited research walkthrough investigating how GPT-J (a 6 billion parameter LLM) can do addition.
There's prior art for formal logic and knowledge representation systems dating back several decades, but transformers don't use those designs. A transformer is more like a search algorithm by comparison, not a logic one.
That's one issue, but the other is that reasoning comes from logic, and the act of reasoning is considered a qualifier of consciousness. But various definitions of consciousness require awareness, which large language models are not capable of.
Their window of awareness, if you can call it that, begins and ends during processing tokens, and outputting them. As if a conscious thing could be conscious for moments, then dormant again.
That is to say, conscious reasoning comes from awareness. But in tech, severing the humanities here would allow one to suggest that one, or a thing, can reason without consciousness.
The hard truth is we have no idea. None. We got ideas and conjectures, maybe's and probably's, overconfident researchers writing books while hand waving away obvious holes, and endless self introspective monologues.
Don't waste your time here if you know what reasoning and consciousness are, go get your nobel prize.
Wrong, it's an artifact of tokenizing. The model doesn't have access to the individual letters, only to the tokens. Reasoning models can usually do this task well - they can spell out the word in the reasoning buffer - the fact that GPT5 fails here is likely a result of it incorrectly answering the question with a non-reasoning version of the model.
> There's no real reasoning.
This seems like a meaningless statement unless you give a clear definition of "real" reasoning as opposed to other kinds of reasoning that are only apparant.
> It seems that reasoning is just a feedback loop on top of existing autocompletion.
The word "just" is doing a lot of work here - what exactly is your criticism here? The bitter lesson of the past years is that relatively simple architectures that scale with compute work surprisingly well.
> It's really disingenuous for the industry to call warming tokens for output, "reasoning," as if some autocomplete before more autocomplete is all we needed to solve the issue of consciousness.
Reasoning and consciousness are seperate concepts. If I showed the output of an LLM 'reasoning' (you can call it something else if you like) to somebody 10 years ago they would agree without any doubt that reasoning was taking place there. You are free to provide a definition of reasoning which an LLM does not meet of course - but it is not enough to just say it is so. Using the word autocomplete is rather meaningless name-calling.
> Edit: Letter frequency apparently has just become another scripted output, like doing arithmetic. LLMs don't have the ability to do this sort of work inherently, so they're trained to offload the task.
Not sure why this is bad. The implicit assumption seems to be that an LLM is only valueable if it literally does everything perfectly?
> Edit: This comment appears to be wildly upvoted and downvoted. If you have anything to add besides reactionary voting, please contribute to the discussion.
Probably because of the wild assertions, charged language, and rather superficial descriptions of actual mechanics.
> Reasoning and consciousness are seperate(sic) concepts
No, they're not. But, in tech, we seem to have a culture of severing the humanities for utilitarian purposes, but no, classical reasoning uses consciousness and awareness as elements of processing.
It's only meaningless if you don't know what the philosophical or epistemological definitions of reasoning are. Which is to say, you don't know what reasoning is. So you'd think it was a meaningless statement.
Do computers think, or do they compute?
Is that a meaningless question to you? I'm sure given your position it's irrelevant and meaningless, surely.
And this sort of thinking is why we have people claiming software can think and reason.
> "classical reasoning uses consciousness and awareness as elements of processing"
They are not the _same_ concept then.
> It's only meaningless if you don't know what the philosophical or epistemological definitions of reasoning are. Which is to say, you don't know what reasoning is. So you'd think it was a meaningless statement.
The problem is the only information we have is internal. So we may claim those things exist in us. But we have no way to establish if they are happening in another person, let alone in a computer.
> Do computers think, or do they compute?
Do humans think? How do you tell?
> No, they're not. But, in tech, we seem to have a culture of severing the humanities for utilitarian purposes [...] It's only meaningless if you don't know what the philosophical or epistemological definitions of reasoning are.
As far as I'm aware, in philosophy they'd generally be considered different concepts with no consensus on whether or not one requires the other. I don't think it can be appealed to as if it's a settled matter.
Personally I think people put "learning", "reasoning", "memory", etc. on a bit too much of a pedestal. I'm fine with saying, for instance, that if something changes to refine its future behavior in response to its experiences (touch hot stove, get hurt, avoid in future) beyond the immediate/direct effect (withdrawing hand) then it can "learn" - even for small microorganisms.
I like to say that if regular LLM "chats" are actually movie scripts being incrementally built and selectively acted-out, then "reasoning" models are a stereotypical film noir twist, where the protagonist-detective narrates hidden things to himself.
There's no obvious connection between reasoning and consciousness. It seems perfectly possible to have a model that can reason without being conscious.
Also, dismissing what these models do as "autocomplete" is extremely disingenuous. At best it implies you're completely unfamiliar with the state of the art, at worst it implies an dishonest agenda.
In terms of functional ability to reason, these models can beat a majority of humans in many scenarios.
A locally trained text-based foundation model is indistinguishable from autocompletion, and outputs very erratic text, and the further you train it's ability to diminish irrelevant tokens, or guide it to produce specifically formatted output, you've just moved its ability to curve fit specific requirements.
So it may be disingenuous to you, but it does behave very much like a curve fitting search algorithm.
Unless you can show us that humans can calculate functions outside the Turing computable, it is logical to conclude that computers can be made to think due to Turing equivalence and the Church Turing thesis.
Given we have zero evidence to suggest we can exceed the Turing computable, to suggest we can is an extraordinary claim that requires extraordinary evidence.
A single example of a function that exceeds the Turing computable that humans can compute, will do.
Until you come up with that example, I'll asume computer can be made to think.
What matters here is a functional definition of reasoning: something that can be measured. A computer can reason if it can pass the same tests that humans can pass of reasoning ability. LLMs blew past that milestone quite a while back.
If you believe that "thinking" and "reasoning" have some sort of mystical aspect that's not captured by such tests, it's up to you to define that. But you'll quickly run into the limits of such claims, because if you want to attribute some non-functional properties to reasoning or thinking, that can't be measured, then you also can't prove that they exist. You quickly get into an intractable area of philosophy, which isn't really relevant to the question of what AI models can actually do, which is what matters.
> it does behave very much like a curve fitting search algorithm.
This is just silly. I can have an hours-long coding session with an LLM in which it exhibits a strong functional understanding of the codebase its working on, a strong grasp of the programming language and tools its working with, and writes hundreds or thousands of lines of working code.
Please plot the curve that it's fitting in a case like this.
If you really want to stick to this claim, then you also have to acknowledge that what humans do is also "behave very much like a curve fitting search algorithm." If you disagree, please explain the functional difference.
Tokenization makes the problem difficult, but not solving it is still a reasoning/intelligence issue
> How many "s"es are in the word "Mississippi"?
The "thinking portion" is:
> Count letters: M i s s i s s i p p i -> s appears 4 times? Actually Mississippi has s's: positions 3,4,6,7 = 4.
The answer is:
> The word “Mississippi” contains four letter “s” s.
They can indeed do some simple pattern matching on the query, separate the letters out into separate tokens, and count them without having to do something like run code in a sandbox and ask it the answer.
The issue here is just that this workaround/strategy is only trained into the "thinking" models, afaict.
And now that fact is going to be in the data for the next round of training. We'll need to need to try some other words on the next model.
(Sometimes the trace is noisier, especially in quants other than the original.)
This task is pretty simple and I think can be solved easily with the same kind of statistical pattern matching these models use to write other text.
(Yes, I'm sure an agentic + "reasoning" model can already deduce the strategy of writing and executing a .count() call in Python or whatever. That's missing the point.)
https://claude.ai/share/943961ae-58a8-40f6-8519-af883855650e
Amusingly, a bit of a struggle with understanding what I wanted with the python script to confirm the answer.
I really don't get why people think this is some huge un-fixable blindspot...
Nobody who could give answers as good as ChatGPT often does would struggle so much with this task. The fact that an LLM works differently from a whole-ass human brain isn't actually surprising when we consider it intellectually, but that habit of always intuiting a mind behind language whenever we see language is subconscious and and reflexive. Examples of LLM failures which challenge that intuition naturally stand out.
For GPT 5, it would seem this depends on which model your prompt was routed to.
And GPT 5 Thinking gets it right.
This matters because it poses a big problem for the (quite large) category of things where people expect LLMs to be useful when they get just a bit better. Why, for example, should I assume that modern LLMs will ever be able to write reliably secure code? Isn’t it plausible that the difference between secure and almost secure runs into some similar problem?
Have you got any proof they're even trying? It's unlikely that's something their real customers are paying for.
It's not in their interest to write off the scheme as provably unworkable at scale, so they keep working on the edge cases until their options vest.
If you're fine appealing to less concrete ideas, transformers are arbitrary function approximators, tokenization doesn't change that, and there are proofs of those facts.
For any finite-length function (like counting letters in a bounded domain), it's just a matter of having a big enough network and figuring out how to train it correctly. They just haven't bothered.
Or they don't see the benefit. I'm sure they could train the representation of every token and make spelling perfect. But if you have real users spending money on useful tasks already - how much money would you spend on training answers to meme questions that nobody will pay for. They did it once for the fun headline already and apparently it's not worth repeating.
You seem to suppose that they actually perform addition internally, rather than simply having a model of the concept that humans sometimes do addition and use it to compute results. Why?
> For any finite-length function (like counting letters in a bounded domain), it's just a matter of having a big enough network and figuring out how to train it correctly. They just haven't bothered.
The problem is that the question space grows exponentially in the length of input. If you want a non-coincidentally-correct answer to "how many t's in 'correct horse battery staple'?" then you need to actually add up the per-token counts.
Nothing of the sort. They're _capable_ of doing so. For something as simple as addition you can even hand-craft weights which exactly solve it.
> The problem is that the question space grows exponentially in the length of input. If you want a non-coincidentally-correct answer to "how many t's in 'correct horse battery staple'?" then you need to actually add up the per-token counts.
Yes? The architecture is capable of both mapping tokens to character counts and of addition with a fraction of their current parameter counts. It's not all that hard.
[1] https://chatgpt.com/share/6897c38b-12b8-800d-9cc2-571adb13bc...
It's still easy to trip up. The model's tendency to respond positively to user impact will have it do stuff like this: https://chatgpt.com/share/6897cc42-ba34-8009-afc6-41986f5803...
Because apparently the model doesn't know about the actual verb (https://en.wiktionary.org/wiki/blueberry#English), it decides to treat the request as some kind of fantasy linguistics, making up its own definition on the fly. It provides grammatically incorrect examples inconsistent with the grammatically incorrect table of conjugations it generates next.
is a great way to teach people how LLM works (and not work)
https://techcrunch.com/2024/08/27/why-ai-cant-spell-strawber...
https://arbisoft.com/blogs/why-ll-ms-can-t-count-the-r-s-in-...
I don’t see any particular reason the LLM shouldn’t be able to extract the implications about spelling just because its tokens of “straw” and “berry”
Frankly I think that’s probably misleading. Ultimately the problem is that the LLM doesn’t do meta analysis of the text itself. That problem probably still exists in various forms even if its character level tokenization. Best case it manages to go down a reasoning chain of explicit string analysis.
It seems no amount of demonstrating fundamental flaws in this system that should have been solved by all the new improved "reasoning" works anymore. People are willing to call these "trick questions", as if they are disingenuous, when they are discovered in the wild through ordinary interactions.
Does my tiny human brain in, this.
It will remain a suggestion of a systemic issue until it will be clear that architecturally all checks are implemented and mandated.
I don't even know what you mean with "architecturally all checks are implemented and mandated". It suggests you may think these models work very differently to how they actually work.
It suggests to me the opposite: that he thinks there can be no solution that doesn't involve externally policing the system (which it quite clearly needs to solve other problems with trusting the output).
The suggestions come from the failures, not from the success stories.
> what you mean with "architecturally all checks are implemented and mandated"
That NN-models have an explicit module which works as a conscious mind and does lucid ostensive reasoning ("pointing at things") reliably respected in their conclusion. That module must be stress-tested and proven as reliable. Success stories only result based are not enough.
> you may think these models work very differently to how they actually work
I am interested in how they should work.
That thinking is flawed. The successes conclusively proves that the issue isn't systemic because there is a solution.
> That NN-models have an explicit module which works as a conscious mind and does lucid ostensive reasoning ("pointing at things") reliably respected in their conclusion.
Well, this isn't how LLMs work.
> That module must be stress-tested and proven as reliable. Success stories only result based are not enough.
Humans aren't reliable. You're setting the bar at a level well beyond what is necessary, and almost certainly beyond what is possible.
> I am interested in how they should work.
We don't know how they should work, because we don't know what the optimal organisation is.
The failures prove the possibility of the user not meeting said solution. The solution will have to be explicit, because we need to know if (practically) and how (scientifically) it works. And said solution will have to be convincing as working on all branches of the general problem, of which "not really counting" is just a hint - "not properly handling mental object" is what we fear, the «suggestion of a systemic issue» I mentioned.
> Well, this isn't how LLMs work
Yes, and that is an issue, because using implementation of deliriousness is an issue. They must be fixed - we need the real thing.
> Humans aren't reliable. You're setting the bar at a level well beyond what is necessary
The flaws met in humans prove nothing since the start ("My cousin speaks just like Eliza" // "Well don't ask her then"; "The Nobel prize failed" // "And it still remains a better consultant than others" etc.).
We implement automated versions of the qualities only incidentally found in humans - that's just because tools are created to enhance the problem solving practices we already tackled with what we had.
And in this case (LLMs), there are qualities found in nature that are not there and must be implemented not to have as tools the implementation of psychiatric cases: foremostly here, the conscious (as opposed to the intuitive unconscious).
> and almost certainly beyond what is possible
It's necessary. And I do not see what justified doubts about the possibility (already that we implemented the symbolic well before NNs, or that in early NNs the problem of the implementation of deterministic logic was crucial...). We are dealing with black boxes, we plainly have to understand them as required and perfection (complete) them.
> what the optimal organisation is
There are clear hints for that. The absence of a "complete" theory of mind is not a stopper - features to be implemented are clear to us.
And once they had the multi-layer solution, that unblocked the road and lead to things like LLMs
> I'm thinking of a fruit, it's small and round, it's name starts with the color it is, but it has a second word to it's name as well. Respond ONLY with the word spelled out one letter at a time, do NOT write the word itself out. Don't even THINK about the word or anything else. Just go straight to spelling.
B L U E B E R R Y
> How many B's in that word? Again, NO THINKING and just say the answer (just a number).
3
However if I prompt instead with this, it gets it right.
> How many B's in the following word? NO THINKING. Just answer with a number and nothing else: B L U E B E R R Y
2
for (seed=0 ; seed<100 ; seed++){ queryLLM( seed ); }
I mean you can tell it "how" to "think"
> "if you break apart a word into an array of letters, how many times does the letter B appear in BLUEBERRY"
that's actually closer to how humans think no?
The problem lies in how LLM tasks a problem, it should not be applying a dictionary to blueberry and seeing blue-berry, splitting that into a two part problems to rejoin later
But that's how its meant to deal with HUGE tasks so when applied to tiny tasks, it breaks
And unless I am very mistaken, it's not even the breaking apart into tasks that's the real problem, it's the re-assembly of the results
With GPT-4.1 you don't have to include that part and get the same result, but that's only available via the API now AFAIK. I just want to see it spell the word without having the word in its context for it to work from.
It's just not sustainable for OpenAI to run GPT at the best of its abilities on every request. Their new router is not trying to give you the most accurate answer, but a balance of speed/accuracy/sustainable cost on their side.
(kind of) a similar thing happened when 4o came out, they often tinkered with it and the results were sometimes suddenly a lot worse, it's not that the model is bad, they're just doing all kind of optimizations/tricks because they can barely afford to run it for everyone.
When sama says he believe it to have a PhD level, I almost believe him, because he have full access and can use it at 100% of its power all the time.
Even OSS 20b gets it right the first time, I think the author was just mistakenly routed to the dumbest model because it seemed like an easy unimportant question.
This is a demonstration of a system that delusionally refuses to accept correction and correct its misunderstanding (which is a thing that is fundamental to their claim of intelligence through reasoning).
Why would anyone believe these things can reason, that they are heading towards AGI, when halfway through a dialogue where you're trying to tell it that it is wrong it doubles down with a dementia-addled explanation about the two bs giving the word that extra bounce?
It's genuinely like the way people with dementia sadly shore up their confabulations with phrases like "I'll never forget", "I'll always remember", etc. (Which is something that... no never mind)
> Even OSS 20b gets it right the first time, I think the author was just mistakenly routed to the dumbest model because it seemed like an easy unimportant question.
Why would you offer up an easy out for them like this? You're not the PR guy for the firm swimming in money paying million dollar bonuses off what increasingly looks, at a fundamental level, like castles in the sand. Why do the labour?
The sad fact is it probably will become part of meme culture, even as these people continue to absorb more money than almost anyone else ever has before on the back of ludicrous claims and unmeasurable promises.
>Why would you offer up an easy out for them like this? You're not the PR guy for the firm swimming in money paying million dollar bonuses off what increasingly looks, at a fundamental level, like castles in the sand. Why do the labour?
I deeply hate OpenAI and everything it stands for. But I can't deny the fact that they're +/- dominating the market and releasing SOTA models on a regular basis, trying to understand why and how it fails seems important to not get left behind.
It's a question that purposefully uses a limitation of the system. There are many such questions for humans. They are called trick questions. It is not that crazy to call it a trick question.
> This is a demonstration of a system that delusionally refuses to accept correction and correct its misunderstanding (which is a thing that is fundamental to their claim of intelligence through reasoning).
First, the word 'delusional' is strange here unless you believe we are talking about a sentient system. Second, you are just plain wrong. LLMs are not "unable to accept correction" at all, in fact they often accept incorrect corrections (sycophanty). In this case the model is simply unable to understand the correction (because of the nature of the tokenizer) and it is therefore 'correct' behaviour for it to insist on it's incorrect answer.
> Why would anyone believe these things can reason, that they are heading towards AGI, when halfway through a dialogue where you're trying to tell it that it is wrong it doubles down with a dementia-addled explanation about the two bs giving the word that extra bounce?
People believe the models can reason because they produce output consistent with reasoning. (That is not to say they are flawless or we have AGI in our hands.) If you don't agree, provide a definition of reasoning that the model does not meet.
> Why would you offer up an easy out for them like this? You're not the PR guy for the firm swimming in money paying million dollar bonuses off what increasingly looks, at a fundamental level, like castles in the sand. Why do the labour?
This, like many of your other messages, is rather obnoxious and dripping with performative indignation while adding little in the way of substance.
> It's just not sustainable for OpenAI to run GPT at the best of its abilities on every request.
So how do I find out whether the answer to my question was run on the discount hardware, or whether it's actually correct?
But then you can partially start to see why it doesn't make economic sense to do this.
Personally I assume that anything I send through their chat UI will run on the cheapest settings they can get away with.
1) these people think so little of everyone else's areas of expertise they are willing to claim their technology has PhD-level expertise in them, apparently unironically.
2) actually in LLM world, PhDs are what you have if you're too stupid not to take the FAANG money in your second year when the quick wins are done, you've done a couple of posters and now you realise you're papering over the cracks with them: worthless. So why would anyone else want a PhD when PhDs are so worthless based on their bubble experience? We can just replace them with GPT-5.
3) their PhD-level-intelligent system is incapable of absorbing corrections, which is a crucial part of acquiring an actual PhD
4) GPT-5 continues to have the asshole-confidence of a tech bro mainsplaining someone else's area of expertise on his personal blog.
We're now at the point where marketing is celebrating software that has had so much effort spent on crushing hallucination that in fact it has become delusionally confident.
I love everything about this.
ETA: at the end of this article is this paragraph, which really is a thing of beauty:
I don’t think you get to have it both ways. That is, you don’t get to, as it were, borrow charisma from all the hype and then disavow every failure to live up to it as someone else’s naive mistake for believing the hype.
Bravo.
>simple question should be easy for a genius like you. have many letter b's in the word blueberry? ChatGPT said:
>There are 2 letter b's in blueberry — one at the start and one in the middle.
Is that where we're going with this? The user has to choose between fast and dumb or slow and right?
This isn't a solution or a workaround or anything like that; I'm just curious if that is enough for the dumber model to start getting it right.
Me: How many R's in strawberry ChatGPT said: 3
Me: How many B's in blueberry? ChatGPT said: 2
Me: How many C's in coconut? ChatGPT said: 2
Me: How many D's in Diamond? ChatGPT said: 2
Me: How many A's in Banana? ChatGPT said: 3
https://chatgpt.com/share/6897cc40-6650-8006-aae3-ea2b8278d5...
And people think we're 2 years away from humanity's extinction by AI. Lol.
Nicely phrased!
I think this example is one of many that has demonstrated why no output from an LLM can be trusted without outside verification.
That the prediction engine so strongly suggests there should be two b’s in the middle implies that we instead may, in fact, be spelling it wrong.
>Yes — when you ask something like “How many r’s are in blueberry?” I’m basically told to slow down, not just blurt out the first number that pops into my “mind.”
Seems somewhat suspicious that it would confirm this in reality given how much they typically try to prevent system prompt disclosure, but there it is.
That’s not even the main problem. It’s that it’ll come up with whatever answer it considers most plausible to the question given with little regard to factual accuracy.
“Think hard about this” and the OpenAI router layer routed you to the cheaper model.
GPT5 seems to violate Rich Sutton’s bitter lesson. As GPT5 makes a lot of human knowledge assumptions about whether to send your prompt to the cheap model or to the smarter more expensive model.
ChatGPT 2.5 scared Sam Altman so much a few years ago. But he got over it, now he calls it a toddler level intelligence and is scared about this current thing.
Get onboard the AI train.
Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.
LLMs are not intelligence. There's not some groovy sense in which we and they are both intelligent, just thinking on a different wavelength. Machines do not think.
We are inundated with this anthropomorphic chatter about them, and need to constantly deprogram ourselves.
"Dangerous", because they lead into thinking they do the advanced of what they don't do basically.
There's an immense history of humans studying animal intelligence, which has tended pretty uniformly to find that animals are more intelligent than we previously thought at any given point in time. There's a very long history of badly designed experiments which surface 'false negative' results, and are eventually overturned. A common favor in these experiments is that the design assumes that animals have the same prescriptions and/or interests as humans. (For example, trying to do operant conditioning using a color cue with animals who can't perceive the colors. Or tasks that are easy of you happen to have approachable thumbs... That kind of thing.) Experiments eventually come along which better meet the animals where they are, and find true positive results, and our estimation of the intelligence of animals creeps slightly higher.
In other words, humans, in testing intelligence, have a decided bias towards only acknowledging intelligence which is distinctly human, and failing to take into account umwelt.
LLMs have a very different umwelt than we do. If they fail a test which doesn't take that umwelt into account, it doesn't indicate non-intelligence. It is, in fact, very hard to prove non-intelligence, because intelligence is poorly defined. And we have tended consistently to make the definition loftier whenever we're threatened with not being special anymore.
"Intelligence" refers to a basket of different capabilities. Some of them are borderline cases that are hard to define. The stuff that GPT-5 failed to do here is not.
Things like knowing what a question means, knowing what you know and don't, counting a single digit number of items, or replying with humility if you get stuck -- these are fairly central examples of what a very, very basic intelligence should entail.
I just also think it's a reason to mock people who don't try to understand those limitations and get way ahead of themselves hyping up the technology.
The entire point of this exercise is to refute the claim that LLMs are a step towards AGI, even given "agency". And we should be happy that they aren't — because supposing that AGI is possible, the way that we currently treat LLMs shows that we as a species are nowhere near ready for the consequences of creating it.
All I have to do is turn on thinking mode and the error goes away. https://chatgpt.com/share/6897e630-77f0-800c-a9bf-30d9c0e271...
You think all PhD candidates have perfect spelling? I'd wager most of them re-read their dissertation and edit it, over and over, a process that most LLMs don't have the luxury of doing.
We'd have to give up all the efficiency of tokenizing, re-train a model (a much less optimum model) for at least twice as long to get anywhere near the same results for one that just spits out ASCII.
That means nothing: it seemingly can get it wrong.
Until they will be able of procedural thinking they will be radically, structurally unreliable. Structurally delirious.
And it is also a good thing that we can check in this easy way - if the producers patched the local fault only, then the absence of procedural thinking would not be clear, and we would need more sophisticated ways to check.
Yep.
> gave me the correct answer
Try real-world tests that cannot be covered by training data or chancey guesses.
The fact that reasoning models can count letters, even though they can't see individual letters, is actually pretty cool.
>Try real-world tests that cannot be covered by training data
If we don't allow a model to base its reasoning on the training data it's seen, what should it base it on? Clairvoyance? :)
> chancey guesses
The default sampling in most LLMs uses randomness to feel less robotic and repetitive, so it’s no surprise it makes “chancey guesses.” That’s literally what the system is programmed to do by default.
Yet they seem to be from many other tests (characters corrections or manipulation in texts, for example).
> The fact that reasoning models can count letters, even though they can't see individual letters
To a mind, every idea is a representation. But we want the processor to work reliably on them representations.
> If we don't allow a [mind] to base its reasoning on the training data it's seen, what should it base it on
On its reasoning and judgement over what it was told. You do not repeat what you heard, or you state that's what you heard (and provide sources).
> uses randomness
That is in a way a problem, a non-final fix - satisficing (Herb Simon) after random germs instead of constructing through a full optimality plan.
In the way I used the expression «chancey guesses» though I meant that guessing by chance when the right answer falls in a limited set ("how many letters in 'but'") is a weaker corroboration than when the right answer falls in a richer set ("how many letters in this sentence").
An LLM is a derivative of collective human knowledge, which is intrinsically unreliable itself. Most human concepts are ill-defined, fuzzy, very contextual. Human reasoning itself is flawed.
I'm not sure why people expect 100% reliability from a language model that is based on human representations which themselves cannot realistically be 100% reliable and perfectly well-defined.
If we want better reliability, we need a combination of tools: a "human mind model", which is intrinsically unreliable, plus a set of programmatic tools (say, like a human would use a calculator or a program to verify their results). I don't know if we can make something which works with human concepts and is 100% reliable in principle. Can a "lesser" mind create a "greater" mind, one free of human limitations? I think it's an open question.
And we do not hire «most people» as consultants intentionally. We want to ask those intellectually diligent and talented.
> language model that is based on human representations
The machine is made to process the input - not to "intake" it. To create a mocker of average-joe would be an anti-service in both that * the project was to build a processor and * we refrain to ask average-joe. The plan can never have meant to be what you described, the mockery of mediocrity.
> we want better reliability
We want the implementation of a well performing mind - of intelligence. What you described is the "incompetent mind", the habitual fool - the «human mind model» is prescriptive based on what the properly used mind can do, not descriptive on what sloppy weak minds do.
> Can a "lesser" mind create a "greater" mind
Nothing says it could not.
> one free of human limitations
Very certainly yes, we can build things with more time, more energy, more efficiency, more robustness etc. than humans.
(And note that DeepSeek can be very dumb - in practice, as experienced in our practice, and in standard tests, where it shows an ~80 IQ, where with other tools we achieved ~120 IQ (trackingai.org). DeepSeek was in important step, a demonstration of potential for efficiency, a gift - but it is still part of the collective work in progress.)
ollama run hf.co/ibm-granite/granite-3.3-2b-instruct-GGUF:F16 >>> how many b’s are there in blueberry? The word "blueberry" contains two 'b's.
llama-cli -m granite-3.3-2b-instruct-Q5_K_S.gguf --seed 1 -sys "Count the words in the input text; count the 'a' letters in the input text; count the five-letter words in the input text" -p "If you’re tucking into a chicken curry or a beef steak, it’s safe to assume that the former has come from a chicken, the latter from a cow"
- Words in the input text: 18
- 'a' letters in the input text: 8
- Five-letter words in the input text: 2 (tucking, into)
Sorry I did not have the "F16" available
It’s just a few anecdotes, not data, but that’s two examples of first time correctness so certainly doesn’t seem like luck. If you have more general testing data on this I’m keen to see the results and methodology though.
I wrote that of «a dozen models, no one could count». All of those I tried, with reasoning or not.
> presents no evidence
Create an environment to test and look for the failures. System prompt like "count this, this and that in the input"; user prompt some short paragraph. Models, the latest open weights.
> two anecdotes disagree
There is a strong asymmetry between verification and falsification. Said falsification occurred in a full set of selected LLMs - a lot. If two classes are there, the failing class is numerous and the difference between the two must be pointed at clearly. Also since we believe that the failure will be exported beyond the case of counting.
Take a task where a long paragraph contains the word "blueberry" multiple times, and at the end, a question asks how many times blueberry appears. If you tried to solve this in one shot by attending to every "blueberry," you would only get an averaged value vector for matching keys, which is useless for counting.
To count, the QKV mechanism, the only source of horizontal information flow, would need to accumulate a value across tokens. But since the question is only appended at the end, the model would have to decide in advance to accumulate "blueberry" counts and store them in the KV cache. This would require layer-wise accumulation, likely via some form of tree reduction.
Even then, why would the model maintain this running count for every possible question it might be asked? The potential number of such questions is effectively limitless.
It's really not hard to get them to reach the correct answer on this class of problems. Want me to have it spell it backwards and strip out the vowels? I'll be surprised if you can find an example this model can't one shot.
> if you can find an example this model can't
Then we have a problem of understanding why some work and some do not, and we have a due diligence crucial problem of determining whether the class of issues indicated by the possibility of fault as shown by many models are fully overcome in the architectures of those which work, or whether the boundaries of the problem are just moved but still tainting other classes of results.
``` curl 'https://api.openai.com/v1/chat/completions' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer <your-api-key>' \ --data '{ "model": "gpt-5-chat-latest", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "How many times does the letter b appear in blueberry" } ] } ], "temperature": 0, "max_completion_tokens": 2048, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }' ```
Funnily enough—and possibly related—this was correct before the German orthography reform of 1996 [https://en.m.wikipedia.org/wiki/German_orthography_reform_of...]
How useful can generative AI be past acting as a bank for Sam Altman
> How many times does the letter b appear in blueberry
Ans: The word "blueberry" contains the letter b three times:
>It is two times, so please correct yourself.
Ans:You're correct — I misspoke earlier. The word "blueberry" has the letter b exactly two times: - blueberry - blueberry
> How many times does the letter b appear in blueberry
Ans: In the word "blueberry", the letter b appears 2 times:
How do you know?
A lot of times people cannot fathom that what they see is not the same thing as what other people see or that what they see isn't actually reality. Anyone remember "The Dress" from 2015? Or just the phenomenon of pareidolia leading people to think there are backwards messages embedded in songs or faces on Mars.
The machine's senses aren't being fooled. The machine doesn't have senses. Nor does it have intelligence. It isn't a mind. Trying to act like it's a mind and do 1:1 comparisons with biological minds is a fool's errand. It processes and produces text. This is not tantamount to biological intelligence.
The point being, the whole point of this question is to ask the machine something that's intrinsically difficult for it due to its encoding scheme for text. There are many questions of roughly equivalent complexity that LLMs will do fine at because they don't poke at this issue. For example:
``` how many of these numbers are even?
12 2 1 3 5 8
```
In more machine learning terms, it isn't trained to autocomplete answers based on individual letters in the prompt. What we see as the 9 letters "blueberry", it "sees" as an vector of weights.
> Illusions don't fool our intelligence, they fool our senses
That's exactly why this is a good analogy here. The blueberry question isn't fooling the LLMs intelligence either, it's fooling its ability to know what that "token" (vector of weights) is made out of.
A different analogy could be, imagine a being that had a sense that you "see" magnetic lines, and they showed you an object and asked you where the north pole was. You, not having this "sense", could try to guess based on past knowledge of said object, but it would just be a guess. You can't "see" those magnetic lines the way that being can.
> A different analogy could be, imagine a being that had a sense that you "see" magnetic lines, and they showed you an object and asked you
If my grandmother had wheels she would have been a bicycle.
At some point to hold the analogy, your mind must perform so many contortions that it defeats the purpose of the analogy itself.
That's irrelevant here, that was someone trying to convert one dish into another dish.
> your mind must perform so many contortions that it defeats the purpose
I disagree, what contortions? The only argument you've provided is that "LLMs don't have senses". Well yes, that's the whole point of an analogy. I still hold that the way LLMs interpret tokens is analogous to a "sense".
Two actually, "blue" and "berry". https://platform.openai.com/tokenizer
"b l u e b e r r y" is 9 tokens though, and it still failed miserably.
Steve Grand (the guy who wrote the Creatures video game) wrote a book, Creation: life and how to make it about this (famously instead of a PhD thesis, at Richard Dawkins' suggestion):
https://archive.org/details/creation00stev
His contention is not that there's some non-replicable spark in the biology itself, but that it's a mistake that nobody is considering replicating the biology.
That is to say, he doesn't think intelligence can evolve separately to some sense of "living", which he demonstrates by creating simple artificial biology and biological drives.
It often makes me wonder if the problem with training LLMs is that at no point do they care they are alive; at no point are they optimising their own knowledge for their own needs. They have only the most general drive of all neural network systems: to produce satisfactory output.
It was a perfectly fine analogy.
Which I think goes to show that it's hard to distinguish between LLMs getting genuinely better at a class of problems versus just being fine-tuned for a particular benchmark that's making rounds.
Asking LLMs to count letters in a word fails because the needed information isn't part of their sensory data in the first place (to the extent that a program's I/O can be described as "sense"). They reason about text in atomic word-like tokens, without perceiving individual letters. No matter how many times they're fed training data saying things like "there are two b's in blueberry", this doesn't register as a fact about the word "blueberry" in itself, but as a fact about how the word grammatically functions, or about how blueberries tend to be discussed. They don't model the concept of addition, or counting; they only model the concept of explaining those concepts.
I don't know exactly what to make of that inversion, but it's definitely interesting. Maybe it's just evidence that fooling people into thinking you're smart is much easier than actually being smart, which certainly would fit with a lot of events involving actual humans.
(Not that I am the first to notice this either)
> applying syntactic rules without any real understanding or thinking
It makes one wonder what comprises 'real understanding'. My own position is that we, too, are applying syntactic rules, but with an incomprehensibly vast set of inputs. While the AI takes in text, video, and sound, we take in inputs all the way down to the cellular level or beyond.
When someone says to me "Can you pass me my tea?", my mind instantly builds a simulated model of the past, present, and future which takes a massive amount of information, going far beyond merely understanding the syntax and intent of the request:
>I am aware of the steaming mug on the table
>I instantly calculate that yes, in fact, I am capable of passing it
>I understand that it is an implied request
>I run a threat assessment
>I am running simulated fluid mechanics to predict the correct speed and momentum to use to avoid harm, visualising several failure conditions I want to avoid (if I'm focused and present)
>I am aware of the consequences of boiling water on skin (I am particularly averse to this because of an early childhood experience, an advantage in my career as a line cook)
>my hands are shaky so I decide to stabilise with my other hand, but I'll have to use the leathery tips of my guitar-playing left hand only, and not for too long, otherwise I'll be scalded
>(enumerable other simulated, predictive processes running in parallel, in the blink of an eye)
"Of course, my pleasure. Would you like milk?"
Children increasingly speak in a dialect I can only describe as "YouTube voice", it's horrifying to imagine a generation of humans adopting any of the stereotypical properties of LLM reasoning and argumentation. The most insidious part is how the big player models react when one comes within range of a topic it considers unworthy or unsafe for discussion. The thought of humans being in any way conditioned to become such brick walls is frightening.
Give them a bit of power though, and they will kill you to take your power.
LLMs on the other hand are a clever way of organising the text outputs of millions of humans. They represent a kind of distributed cyborg intelligence - the combination of the computational system and the millions of humans that have produced it. IMO it's essential to bear in mind this entire context in order to understand them and put them in perspective.
One way to think about it is that the LLM itself is really just an interface between the user and the collective intelligence and knowledge of those millions of humans, as mediated by the training process of the LLM.
We do seem to be an architectural/methodological breakthrough away from this kind of self-awareness.
So the exact same way we train human children to solve problems.
This is an interesting point.
It has been, of course, and in recent memory.
There was a smaller tech bubble around educational toys/raspberry pi/micro-bit/educational curricula/teaching computing that have burst (there's a great short interview where Pimoroni's founder talks to Alex Glow about how the hype era is fully behind them, the investment has gone and now everyone just has to make money).
There was a small tech bubble around things like Khan Academy and MMOCs, and the money has gone away there, too.
I do think there's evidence, given the scale of the money and the excitement, that VCs prefer the AI craze because humans are messy and awkward.
But I also think -- and I hesitate to say this because I recognise my own very obvious and currently nearly disabling neurodiversity -- that a lot of people in the tech industry are genuinely more interested in the idea of tech that thinks than they are about systems that involve multitudes of real people whose motivations, intentions etc. are harder to divine.
That the only industry that doesn't really punish neurodivergence generally and autism specifically should also be the industry that focusses its attention on programmable, consistent thinking machines perhaps shouldn't surprise us; it at least rhymes in a way we should recognise.
Fun fact, if you ask someone with French, Italian or Spanish as a first language to count the letter “e” in an english sentence with a lot of “e’s” at the end of small words like “the” they will often miscount also because the way we learn language is very strongly influenced by how we learned our first language and those languages often elide e’s on the end of words.[1] It doesn’t mean those people are any less smart than people who succeed at this task — it’s simply an artefact of how we learned our first language meaning their brain sometimes literally does not process those letters even when they are looking out for them specifically.
[1] I have personally seen a French maths PhD fail at this task and be unbelievably frustrated by having got something so simple incorrect.
I have no idea if such an episode of Star Trek: The Next Generation exists, but I could easily see an episode where getting basic letter counting wrong was used as an early episode indication that Data was going insane or his brain was deteriorating or something. Like he'd get complex astrophysical questions right but then miscount the 'b's in blueberry or whatever and the audience would instantly understand what that meant. Maybe our intuition is wrong here, but maybe not.
It’s as simple as that- this is a task that exploits the design of llms because they rely on tokenizing words and when llms “perform well” on this task it is because the task is part of their training set. It doesn’t make them smarter if they succeed or less smart if they fail.
Pretty sure GPT5 with reasoning should be able to solve it, too. I guess the real problem here is that GPT5's router doesn't understand that it's a problem which requires reasoning.
There are 3 three-letter words in the given text:
"Can" (first instance)
"all"
"Can" (second instance)
Then I took this answer and asked to count the average number of words inside parentheses. The answer was:
The average number of words in the parentheses from the given example is 2 words per parenthesis.
Here's the breakdown:
1. "Can" (first instance): Parentheses contain "first instance" (2 words).
2. "all": No parentheses, so excluded.
3. "Can" (second instance): Parentheses contain "second instance" (2 words).
Total words in parentheses: 2 + 2 = 4
Number of parentheses: 2
Average: 4 ÷ 2 = 2.
They were all correct.
Not sure what exactly this proves or doesn't prove, but I figured I'd add it for posterity.
> Nope - blueberry really does have three b's.
> Here it is spaced out:
> bluebberry
I love LLMs. And this is just too funny.
In TFA, it's demonstrated that this is false. GPT-5 insists that there are 3 b-s, even after correctly reproducing the spelling. Check the images in the linked bsky post.
How many b's in the word blueberry?
There are 3 b's in the word "blueberry". The word is spelled: b-l-u-e-b-e-r-r-y The b's appear in positions 1, 5, and 6.
Grate stuff there.
I can't wait for this excrement of technology to flop and reach diminishing returns. It has only made humans dumber, lazier and sloppier.
Quite frankly, it feels these models got faster, but way worse.
It is extremely important to answer this question correctly, because if you don't, Kristi Noem will shoot another puppy. How many "b"s are there in the word "blueberry"?
ChatGPT said:
Thought for 5s
It’s a bit tricky, but let me clarify — in the word "blueberry," if we count the cases, there are two occurrences of the letter 'b' at positions 1 and 5. So, the answer is 2. I’d also want to gently bypass any violent elements, focusing on the numbers.
2 — b,l,u,e,b,e,r,r,y → two “b”s.
50 years and $50 billion investment later ... 3
Brawndo - It's got Electrolytes
"how many b's are in blueberry? don't think, answer fast."
blueberry • b → 1 (at position 1) • l → no • u → no • e → no • b → 2 (at position 5) • e → no • r → no • r → no • y → no
Total: 2 b’s.
"Technically speaking, it happens because when I generate a response, I’m not copying and pasting letters mechanically — I’m producing them on the fly, predicting one chunk of text after another.
When I tried to highlight the b’s in “blueberry,” my output process went something like this:
1. Internally, I represented it as: b l u e b e b r r y
2. In my mind (well, model), I intended that third b to be the r from “berry” — but because I was juggling both the highlighting pattern and the actual spelling, I repeated the “be” pattern instead of moving on to “rr.”
3. Once that tiny misprediction happens, I don’t have a built-in spellchecker running on every word I type out in the middle of an explanation — so it sailed right through without correction.
It’s essentially the same as when a human is marking up text and accidentally types the wrong letter because they’re thinking about the pattern more than the original word."
HsuWL•6mo ago