This is pretty ironic, considering the subject matter of that blog post. It's a super-common misconception that's gained very wide popularity due to reactionary (and, imo, rather poor) popular science reporting.
The author parroting that with confidence in a post about Dunner-Krugering gives me a bit of a chuckle.
ANNs are arbitrary function approximators. The training process uses statistical methods to identify a set of parameters that approximate the function as best as possible. That doesn't necessarily mean that the end result is equivalent to a very fancy multi-stage linear regression. It's a possible outcome of the process, but it's not the only possible outcome.
Looking at a LLMs I/O structure and training process is not enough to conclude much of anything. And that's the misconception.
I'm not sure I follow. LLMs are probabilistic next-token prediction based on current context, that is a factual, foundational statement about the technology that runs all LLMs today.
We can ascribe other things to that, such as reasoning or knowledge or agency, but that doesn't change how they work. Their fundamental architecture is well understood, even if we allow for the idea that maybe there are some emergent behaviors that we haven't described completely.
> It's a possible outcome of the process, but it's not the only possible outcome.
Again, you can ascribe these other things to it, but to say that these external descriptions of outputs call into question the architecture that runs these LLMs is a strange thing to say.
> Looking at a LLMs I/O structure and training process is not enough to conclude much of anything. And that's the misconception.
I don't see how that's a misconception. We evaluate all pretty much everything by inputs and outputs. And we use those to infer internal state. Because that's all we're capable of in the real world.
I think the reason people don't say that is because they want to say "I already understand what they are, and I'm not impressed and it's nothing new". But what the comment you are replying to is saying is that the inner workings are the important innovative stuff.
LLMs are probabilistic or non-deterministic computer programs, plenty of people say this. That is not much different than saying "LLMs are probabilistic next-token prediction based on current context".
> I think the reason people don't say that is because they want to say "I already understand what they are, and I'm not impressed and it's nothing new". But what the comment you are replying to is saying is that the inner workings are the important innovative stuff.
But we already know the inner workings. It's transformers, embeddings, and math at a scale that we couldn't do before 2015. We already had multi-layer perceptrons with backpropagation and recurrent neural networks and markov chains before this, but the hardware to do this kind of contextual next-token prediction simply didn't exist at those times.
I understand that it feels like there's a lot going on with these chatbots, but half of the illusion of chatbots isn't even the LLM, it's the context management that is exceptionally mundane compared to the LLM itself. These things are combined with a carefully crafted UX to deliberately convey the impression that you're talking to a human. But in the end, it is just a program and it's just doing context management and token prediction that happens to align (most of the time) with human expectations because it was designed to do so.
The two of you seem to be implying there's something spooky or mysterious happening with LLMs that goes beyond our comprehension of them, but I'm not seeing the components of your argument for this.
Overconfident and wrong.
No one understands how an LLM works. Some people just delude themselves into thinking that they do.
Saying "I know how LLMs work because I read a paper about transformer architecture" is about as delusional as saying "I read a paper about transistors, and now I understand how Ryzen 9800X3D works". Maybe more so.
It takes actual reverse engineering work to figure out how LLMs can do small bits and tiny slivers of what they do. And here you are - claiming that we actually already know everything there is to know about them.
What more are LLMs than statistical inference machines? I don't know that I'd assert that's all they are with confidence but all the configurations options I can play with during generation (Top K, Top P, Temperature, etc.) are all ways to _not_ select the most likely next token which leads me to believe that they are, in fact, just statistical inference machines.
> As I ChatGPT user I notice that I’m often left with a sense of certainty.
They have almost the opposite effect on me.
Even with knowledge from books or articles I've learned to multi-source and question things, and my mind treats the LLMs as a less reliable averaging of sources.
If you tell it to find something and give it a detailed description of what you're looking for, it will pretend like it has verified that that thing exists, and give you a bulletpoint lecture about why it is such an effective and interesting thing that 1) you didn't ask for, and 2) is really it parroting your description back to you with embellishments.
I thought I was going to be able to use LLMs primarily for research, because I have read an enormous number of things (books, papers) in my life, and I can't necessarily find them again when they would be useful. Trying to track them down through LLMs is rarely successful and always agonizing, like pulling teeth that are constantly lying to you. A surprising outcome is that I often get so frustrated by the LLM and so detailed in how I'm complaining about its stupid responses that I remind myself of something that allows me to find the reference on my own.
I have to suspect that people who find it useful for research are researching things that are easily discoverable through many other means. Those are not the things that are interesting. I totally find it useful to find something in software docs that I'm too lazy to look up myself, but it's literally saving me 10 minutes.
"Don't just trust wikipedia, check it's resources, because it's crowdsourced and can be wrong".
Now, almost 2 decades later, I rarely hear this stance and I see people relying on wikipedia as an authoritative source of truth. i.e, linking to wikipedia instead of the underlying sources.
In the same sense, I can see that "Don't trust LLMs" will slowly fade away and people will blindly trust them.
I also can't even read the second sentence. I think there are typos there, but there's no mental correction I can do to make it coherent for me.
I find I trust Wikipedia less these days, though still more than LLM output.
Using Perplexity or Claude in "please source your answer" mode is much more like a conventional search engine than looking up data embedded in 5 trillion (or whatever) parameters.
Ive noticed things like gemini summaries on Google searches are also generally close enough.
That's a different scenario. You shouldn't _cite wikipedia in a paper_ (instead you should generally use its sources), but it's perfectly fine in most circumstances to link it in the course of an internet argument or whatever.
This comes from decades of teachers misremembering what the rule was, and eventually it morphed into the Wikipedia specific form we see today - the actual rule is that you cannot cite an encyclopaedia in an academic paper. full stop.
Wikipedia is an encyclopaedia and therefore should not be cited.
Wikipedia is the only encyclopaedia most people have used in the last 20 years, therefore Wikipedia = encyclopaedia in most people's minds.
There's nothing wrong with using an encyclopaedia for learning or introducing yourself to a topic (in fact this is what teachers told students to do). And there's nothing specifically wrong about Wikipedia either.
Today they might have better trust for Wikipedia-- and I know I use it as a source of truth for a lot of things-- but back in my day teachers were of the opinion that it couldn't be trusted. This was for like middle and high school, not college or university, so we would cite encyclopedias and that sort of thing, since we weren't reading cutting edge papers back then (maybe today kids read them, who knows).
Edit: Also, I think the GP comment was proven correct by all of the replies claiming that Wikipedia was never controversial because it was very clear to everyone my age when Wikipedia was created/founded that teachers didn't trust the internet nor Wikipedia at the time.
All of that said, I trust Wikipedia more than I trust any LLMs but don't rely on either as a final source for understanding complex topics.
Isn't that exactly what happens on any controversial Wikipedia page?
An encyclopedia does not cite its sources, and does not claim to be a primary source, its potentialymistakes cannot be checked.
(Wikipedia has the additional problem that, by default, the version cited is the ever-changing "latest" version, not a fixed and identified version.)
I do understand the "latest version" argument, and it is a weakness, but it's also a double edged sword - it means Wikipedia can also be more up-to-date than (almost) any other source for the information. Thats why I say there's "nothing specifically wrong about Wikipedia either" it can be held in similar regard to other tertiary sources and encyclopaedias - with all the problems that come with those.
Either way, you are correct, we weren't allowed to cite any encyclopedia, but they were meant to be jumping off points for papers. After Wikipedia launched when I was in 9th grade, we weren't allowed to even look at it (blocked from school computers).
I definitely used it though.
In turn-of-the-century boomer world, reality was what you saw on TV. If you saw something with your own eyes that contradicted the world view presented by the media, then one's eyes were to be disbelieved. The only reputable sources of news were the mainstream media outlets. The only credible history books would be those with reviews from the mainstream media, with anything else just being the 'ramblings of a nutter'.
In short, we built a beautiful post-truth world and now we are set on outsourcing our critical thinking to LLMs.
That's already happening. I don't even think we had a very long "Don't trust LLMs" phase, if we did it was very short.
The "normies" already trust whatever they spit out. At leadership meetings at my work, if I say anything that goes against the marketing hype for LLMs, such as talking about "Don't trust LLMs", it's met with eye rolls and I'm not forward thinking enough, blah blah.
Management-types have 100% bought into the hype and are increasingly more difficult to convince otherwise.
Yeah, the stupid.
I've been thinking about this a bit. We don't really think this way in other areas, is it appropriate to think this way here?
My car has an automatic transmission, am I a fraud because the machine is shifting gears for me?
My tractor plows a field, am I a fraud because I'm not using draft horses or digging manually?
Spell check caught a word, am I a fraud because I didn't look it up in a dictionary?
And, for instance, I have barely any knowledge of how my computer works, but it's a tool I use to do my job. (and to have fun at home.)
Why are these different than using LLMs? I think at least for me the distinction is whether or not something enables me to perform a task, or whether it's just doing the task for me. If I had to write my own OS and word processor just to write a letter, it'd never happen. The fact that the computer does this for me facilitates my task. I could write the letter by hand, but doing it in a word processor is way better. Especially if I want to print multiple copies of the letter.
But for LLMs, my task might be something like "setting up apache is easy, but I've never done it so just tell me how do it so I don't fumble through learning and make it take way longer." The task was setting up Apache. The task was assigned to me, but I didn't really do it. There wasn't necessarily some higher level task that I merely needed Apache for. Apache was the whole task! And I didn't do it!
Now, this will not be the case for all LLM-enabled tasks, but I think this distinction speaks to my experience. In the previous word processor example, the LLM would just write my document for me. It doesn't allow me to write my document more efficiently. It's efficient, only in the sense that I no longer need to actually do it myself, except for maybe to act as an editor. (and most people don't even do much of that work) My skill in writing either atrophies or never fully develops since I don't actually need to spend any time doing it or thinking about it.
In a perfect world, I use self-discipline to have the LLM show me how to set up Apache, then take notes, and then research, and then set it up manually in subsequent runs; I'd have benefited from learning the task much more quickly than if I'd done it alone, but also used my self-discipline to make sure I actually really learned something and developed expertise as well. My argument is that most people will not succeed in doing this, and will just let the LLM think for them.
So, while it's an imperfect answer that I haven't really nailed down yet, maybe the answer is just to realize this and make sure we're doing hard things on purpose sometimes. This stuff has enabled free time, we just can't use it to doomscroll.
That's an interesting take on the loneliness crisis that I had not considered. I think you're really onto something. Thanks for sharing. I don't want to dive into this topic too much since it's political and really off-topic for the thread, but thank you for suggesting this.
Cars help people be social in my world. I would say that riding on a train in your own bubble with strangers is not a social activity, but others would disagree.
[1]https://www.bunkhistory.org/resources/when-did-tv-watching-p...
I would say that with a computer you're using a tool to take care of mundane details and speed up the mechanics of tasks in your life. Such as writing a document, or playing a game. I can't think of a way I would be seriously disadvantaged by not having the ability to hand-write an essay or have games I can readily play without a computer. Computers are more like tools in the way a hammer is a tool. I don't mind being totally dependent on a computer for those tasks in the same way I don't mind that I need a hammer anytime I want to drive a nail.
But for many people, LLMs replace critical thinking. They offer the allure of outsourcing planning, research, and generating ideas. These skills seem more fundamental to me, and I would say there's definitely a loss somehow of one's humanity if you let those things atrophy to the point you become utterly dependent on LLMs.
You don't just set up Apache to have run Apache? You set it up to serve web content! It is middleware, it is not in of itself useful?
Isn't setting up Apache robbing yourself of the opportunity to learn about writing your own HTTP server? In C? And what a bad idea that is?
The LLM helping you configure a web server is no different than the web server helping you serve HTTP instead of implementing a web server from scratch. You've just seemingly? arbitrarily decided your preferred abstraction layer is where "real work" happens.
Okay, maybe LLMs might disappear tomorrow and so for some reason the particular skill of configuring Apache will become useful again, maybe! But I'm already using brainpower to memorize phone numbers in case my smartphone contacts disappear, so maybe I won't have room for those Apache configs ;-)
I don't want my software developer's experience to turn into a real estate developer's experience. I don't want to go from being a technical knowledge worker to a financier or contract negotiator. I've realized I was never in it for the outcome. I was in it for the exploration and puzzles.
Similarly, I don't want to become a "Hollywood producer" cliche. This caricature was a common joke earlier in my tech career in Southern California. We detested the idea of becoming a "tech" person acting like a Steve Martin parody of a Hollywood wheeler-dealer. Someone sitting in a cafe, pitching ideas that was nothing more than a reference to an existing work with an added gimmick or casting change.
To me, that caricature combines two negative aspects. One is the heavily derivative and cynical nature. The other is the stratospheric abstraction level, where folks at this level see themselves as visionaries rather than just patrons of someone else doing all the creative work.
I don't want to be a patron of an LLM or other black box.
With the caveat of for our field in particular, it's one of the few that require continuous learning and adaptation, so tech workers in a way are better predisposed to this line of thinking and tool adoption without some of the potential harmful side effects.
To pick on spell check, it has been showing that we can develop a dependency on it and thereby losing our own ability to spell and reason about language. But, is that a bad thing? I don't know.
What I do know is humans have been outsourcing our thinking for a long time. LLMs are another evolution in that process, just another way to push off cognitive load onto a tool like we've done with stone tablets, books, paper notes, digital notes, google, etc.
The jury is Still out on what value these things will bring
Some people certainly seem to be. You see this a lot on webforums; someone spews a lot of confident superficially plausible-looking nonsense, then when someone points out that it is nonsense, they say they got it from a magic robot.
I think this is particularly common for non-tech people, who are more likely to believe that the magic robots are actually intelligent.
Lots of folks think it's amazing and greatly empowers them.
I work for a company with an open source product and the number of support requests we get from people who ask the chatbot to do their config and then end up with something nonfunctioning is quite significant. Goes up to users complaining our api is down because the chatbot hallucinated the endpoint.
It is not.
Is it me or does everyone find that dumb people seem to use this statement more than ever?
We are all geniuses!
This is not what the Dunning-Kruger effect is. It's lacking metacognitive ability to understand one's own skill level. Overconfidence resulting from ignorance isn't the same thing. Joe Rogan propagated the version of this phenomenon that infiltrated public consciousness, and we've been stuck with it ever since.
Ironically, you can plug this story into your favorite LLM, and it will tell you the same thing. And, also ironically, the LLM will generally know more than you in most contexts, so anyone with a degree epistemic humility is better served taking it at least as seriously as their own thoughts and intuitions, if not at face value.
LLMs are cool and useful technology, but if you approach them with the attitude you're talking with an other, you are leaving yourself vulnerable to all sorts of cognitive distortions.
The larger problem is cognitive offloading. The people for whom this is a problem were already not doing the cognitive work of verifying facts and forming their own opinions. Maybe they watched the news, read a Wikipedia article, or listened to a TEDtalk, but the results are the same: an opinion they felt confident in without a verified basis.
To the extent this is on 'steroids', it is because they see it as an expert (in everything) computer and because it is so much faster than watching a TED talk or reading a long form article.
Provide a person confidence in their opinion and they will not challenge it, as that would risk the reward of lend you live in a coherent universe.
The majority person has never heard the term “epistemology” despite the concept being central to how people derive coherence. Yet all these trite pieces written about AI and its intersectionality with knowledge claim some important technical distinction.
I’m hopeful that a crisis of epistemology is coming, though that’s probably too hopeful. I’m just enjoying the circus at this point
Regardless of what media you get your info from you have to be selective of what sources you trust. It's more true today than ever before, because the bar for creating content has never been lower.
That's bad when their output can be complete garbage at times.
And worse of all is that it's not even consistent, because I tried the same searches again and I couldn't get the same answer, so it just randomly decides to assert complete nonsense sometimes while other times it gives the right answer or says something completely unrelated.
It's really been a major negative in my search experience. Every time I search for something I can't be sure that it's actually quoting anything verbatim, so I need to check the sources anyway. Except it's much harder to find the link to the source with these AI's than it is to just browse the verbatim snippets in a simple list of search results. So it's just occupying space with something that is simply less convenient.
You can go through and look at the websites it checked, and it's 80% blogspam with no other sources cited on said blog.
When I'm manually doing a Google search, I'm not just randomly picking the first few links I'm deliberately filtering for credible domains or articles, not just picking whatever random marketing blog SEO'd their way to the top.
Sorry Gemini, an Advertorial from Times of India is not a reliable source for what I'm looking for. Nor is this xyz affiliate marketing blog stuffed to the brim with ads and product placement.
Some of that is due to that's probably 90% of the internet, but weren't these things trained on huge amounts of books, and published peer-reviewed works? Where are those in the sources?
When I use chatGPT I do the same before I've asked for the fact: how common is this problem? how well known is it? How likely is that chatgpt both knows it and can surface it? Afterwards I don't feel like I know something, I feel like I've got a faster broad idea of what facts might exist and where to look for them, a good set of things to investigate, etc.
This more closely fits our models of cognition anyway. There is nothing really very like a filter in the human mind, though there are things that feel like them.
The fact that LLMs seem like people but aren't, specifically have a lot of the signals of a reliable source in some ways, I'm not sure how these processes will map. I'm skeptical of anyone who is confident about it in either way, in fact.
> The mental motion of “I didn’t really parse that paragraph, but sure, whatever, I’ll take the author’s word for it” is, in my introspective experience, absolutely identical to “I didn’t really parse that paragraph because it was bot-generated and didn’t make any sense so I couldn’t possibly have parsed it”, except that in the first case, I assume that the error lies with me rather than the text. This is not a safe assumption in a post-GPT2 world. Instead of “default to humility” (assume that when you don’t understand a passage, the passage is true and you’re just missing something) the ideal mental action in a world full of bots is “default to null” (if you don’t understand a passage, assume you’re in the same epistemic state as if you’d never read it at all.)
https://www.greaterwrong.com/posts/4AHXDwcGab5PhKhHT/humans-...
Can you cite a specific example where this happened for you? I'm interested in how you think you went from "broad idea" to building actual knowledge.
To make his point, you need specific examples from specific LLMs.
[1] https://www.mcgill.ca/oss/article/critical-thinking/dunning-...
The term I’ve been using of late is “authority simulator.” My formative experiences with “authority figures” was a person who can speak with breadth and depth about a subject and who seems to have internalized it because they can answer quickly and thoroughly. Because LLMs do this so well, it’s really easy to feel like you’re talking to an authority in a subject. And even though my brain intellectually knows this isn’t true, emotionally, the simulation of authority is comforting.
Quantity has a quality of its own. The first chess engine to beat Gary Kasparov wasn't fundamentally different than earlier ones--it just had a lot more compute power.
The original Google algorithm was trivial: rank web pages by incoming links--its superhuman power at giving us answers ("I'm feeling lucky") was/is entirely due to a massive trove of data.
And remember all the articles about how unreliable Wikipedia was? How can you trust something when anyone can edit a page? But again, the power of quantity--thousands or millions of eyeballs identifying errors--swamped any simple attacks.
Yes, LLMs are literally just matmul. How can anything useful, much less intelligent, emerge from multiplying numbers really fast? But then again, how can anything intelligent emerge from a wet mass of brain cells? After all, we're just meat. How can meat think?
Some of us used to think that meat spontaneously generated flies. Maybe someday we'll (re-)learn that meat doesn't spontaneously generate thought either?
And especially not now. Not when LLMs can already do pretty much anything that a human can - and some of those things they can even do well.
But I always resist the urge. Because I think: Isn't it always going to have some kinds of people like that? With or without this LLM thing.
If there is anything to hate about this technology, for the more and more bullshits we see/hear in daily life, it is: (1) Its reach: More people of all ages, of different backgrounds, expertise, and intents are using it. Some are heavily misusing it. (2) Its (ever increasing) capability: Yes, it has already become pretty easy for ChatGPT or any other LLMs to produce a sophisticated but wrong answer on a difficult topic. And I think the trend is that with later, more advanced versions, it would become harder and take more effort to spot a hidden failure lurking in a more information-dense LLM's answer.
Despite, LLM's are useful. I could write the code faster without an LLM, but then I'd have code that wasn't carefully reviewed line-by-line because my coworkers trust me (the fools). It'd have far fewer tests because nobody forced me to prove everything. It'd have worse naming because every once in a while the LLM does that better than me. It'll be missing a few edge cases the LLM thought of that I didn't. It'd have forest/trees problems because if I was writing the code I'd be focused on the code instead of the big picture.
I'm not sure this statement is true most of the time. This kind of reasoning reminds me of the discussion around 'code correctness'. In my opinion there are very few instances where correctness is really important. Most of the time you just need something that works well enough.
Imagine you have a continuous numeric scale that goes from 'never works' to '100% formal proofs' to indicate the correctness of every piece of software. Pushing your code to the '100% formal proofs' side takes a lot of resources, that could be deployed on other places.
Typically, for the last 2 years, I don't feel that anyone can or bother to read anymore.
this is so wrong! i simply can't get ChatGPT to admit something clearly wrong. it can play both sides and gives nuance which is exactly what i expect. but it is so un-sycopanthic that it won't leave you feeling like you are right. any examples of it doing so are welcome! show me examples where it takes a clearly wrong or false idea and makes it look as if it is a good idea (unless you specifically ask it to do it).
Why is the "Dunning-Kruger" is not mentioned anywhere in the article body while is gloriously visible in the title? By the way, AI is not wrong "all the time".
for example: people fall in love all the time.
Prompt: Compare the Space episode of Bluey to Interstellar
Response: That’s a really fun and surprisingly fitting comparison! Let’s break down how Bluey’s “Space” episode parallels Christopher Nolan’s Interstellar — both emotionally and thematically — even though one is a 7-minute preschool cartoon and the other a 3-hour sci-fi epic.
1. Core Theme: Love Across Distance
Interstellar: The film centers on a father (Cooper) who leaves his daughter (Murph) to save humanity, exploring how love transcends time and space — literally. The emotional heart of Interstellar is the bond between parent and child that endures despite impossible separation.
Bluey: “Space” In this episode, Bluey, Bingo, and their dad (Bandit) play an imaginative game where they’re astronauts who have to separate to save their mission. The moment Bluey and Bandit must say goodbye echoes that same emotional tension — the pain of letting go but also the love that motivates sacrifice.
Parallel: Both stories use the setting of outer space as a metaphor for emotional distance — how people we love can feel far away, and how connection can bridge that gap.
It makes sense to refer to it as a concept but it's probably not an appropriate assumption to make about people.
[0] https://www.mcgill.ca/oss/article/critical-thinking/dunning-...
Maybe this is my misunderstanding but I don't think the common invocation really took it as a law that the unknowledgeable always think their skills are higher.
There are a lot of office jobs that I'd fit into the category of "bullshit jobs." They may serve some purpose in the huge bureaucracy of enterprises but the day to day ultimately boils doing to managing someone's calendar and sending emails.
Quite a few people at my work have now started using Copilot for their emails. It's obviously AI (at least to me), and yet, the content and formatting are an improvement over what they were sending before.
So much of the marketing hype on LLMs is about how it'll replace all the engineering work (the MBA's wet dream, to replace all the expensive labor). In reality, I think its more capable at replacing non-tech labor and middle management.
An LLM can send out an email to the team and analyze a project check-in faster, and better, than some overpaid middle manager can. I have no doubts an LLM could probably serve the role of a project management office, or a business analyst.
Sure, there should still be a human in the loop for now, but you need far, far less humans in those roles than previously.
I was quite impressed that it didn't "give in" and validate my wrong idea.
Next, I hope we can somehow get LLMs to distinguish between reliable and less-reliable results.
Edit: it was https://christianheilmann.com/2025/10/30/ai-is-dunning-kruge...
Coding in the Red-Queen Era - Don't Let Agents Eat Your Brain https://corecursive.com/red-queen-coding/
You now see threads on X of famous people using Grok to explain how smart their ideas are. But there’s a problem: You can literally get it to do that with every single dumb idea.
The Gell-Mann Amnesia Effect https://en.wikipedia.org/wiki/Gell-Mann_amnesia_effect
For example, I just bought a 1990 Miata and I want to install a couple of rocker switches in the dash to individually control the pop-up headlights. I have enough circuits knowledge to safely change outlets and light switches, but I didn't know about relays. I asked ChatGPT how to add these switches and it immediately mentioned buying DPDT switches and tying in the OEM relay into a SPDT relay. It may have gotten the actual circuit diagram completely wrong, but now I know exactly what to read up on.
Wow, I have a couple Bill Bryson books on my reading list, can you share some examples of that?
Every time I start getting lazy and asking ChatGPT things like "write me a singleton that tracks progression for XYZ in a unity project", I wind up with a big hole where some deeper understanding of my problem should be. A better approach is to prompt it like "Show me a few ways to persist progression-like data in a unity project. Compare and contrast them".
Having an LLM development policy where you ~blindly accept a solution simply because it works is like an HOV lane to hell. It is very tempting to do this when you are tired or in a rush. I do it all the time.
It wasn't until working through practice problems later, on my own, did it become clear how much detail I was missing.
This is a common problem in learning. Recognition is easier than recall and smoothness is confused for understanding.
You actually need to struggle with the concepts a bit to learn effectively. Without the struggle it feels more effective, but is not.
I don't have a problem just ignoring all the LLM glazing, although I'd really like the ability to turn it off.
The fact that they've all been trained to do it, because so many of the "normies" fall for it, is kind of an indictment in my eyes. Bit of a mirror held up to society.
You should probably be worried about how fake flattery works so well in society, and how this enables sociopaths and narcissists to flourish and control everything.
This LLM problem is just a symptom.
It presented a reasonable agenda, however 80% of the rockhounding spots were completely made up!
Over time, and as LLMs have gotten less sycophantic, I’ve found myself trusting them a bit more (a dangerous and slippery slope).
With that said, GPT-4o in particular, seemed to rank user satisfaction above truth.
I’ve found that GPT-5 Pro is currently the best at pushing back against silly ideas, and does a decent job of informing me that my questions could be better (:
Brendinooo•4h ago
This is a good line, and I think it tempers the "not just misinformed, but misinformed with conviction" observation quite a bit, because sometimes moving forward with an idea at less than 100% accuracy will still bring the best outcome.
Obviously that's a less than ideal thing to say, but imo (and in my experience as the former gifted student who struggles to ship) intelligent people tend to underestimate the importance of doing stuff with confidence.
shermantanktop•3h ago
Seeing others get burned by that pattern over and over can encourage hesitation and humility, and discourage confident action. It’s essentially an academic attitude and can be very unfortunate and self-defeating.