Here is the original DK article: https://pubmed.ncbi.nlm.nih.gov/10626367/
Turns out I thought that the author was DKing about DK, but actually I was DKing about them DKing about DK.
Original Comment:
I have high-confidence in a nitpick, and low-confidence in a reason to think this thesis is way off.
The Nitpick:
Dunning-Kruger effect is more about how confidence and competence evolve over time. It's how when we learn an overview about our new topic our confidence (in understanding) greatly exceeds our competence, then we learn how much we don't know and our confidence crashes below our actual competence, and then eventually, when we reach mastery, they become balanced. The dunning-Kruger effect is this entire process, not only the first part, which is colloquially called "Peak Mt Stupid" after the shape of the confidence vs competence graph over time.
The Big Doubt:
I can't help but wonder if fools asking AI questions and getting incorrect answers and thinking they are correct is some other thing all together. At best maybe tangentially related to DK.
After reading your comment I navigated to it directly and found the first two sentences:
The Dunning–Kruger effect is a cognitive bias that describes the systematic tendency of people with low ability in a specific area to give overly positive assessments of this ability. The term may also describe the tendency of high performers to underestimate their skills.
Unsatisfied that this was the authority, I dug up the original paper here:
https://pubmed.ncbi.nlm.nih.gov/10626367/
And sure enough, the emphasis in the abstract is exactly as you say.
So while I stand by that the principle behind the "effect" is how confidence and competence evolve over time as someone discovers and masters a domain, I will concede that the original authors, and most people, assign the name for it to primarily the first phase.
Here I was thinking the OP Author was DKing about DK, but in reality I was DKing about them DKing about DK.
What research is this based on?
However the over-time aspect is kind of self-evident. No one is born a master of any skill or subject.
So while the original research was based on a snapshot of many people along a developmental journey, just because the data collection wasn't done over time doesn't mean that the principle behind the effect isn't time/effort based.
The graph is really competence vs confidence, but competence can only increase over time. If this isn't self-evident enough, there is lots of research on how mastery is gained. I don't have time to throw a bunch of it at you, but I suspect you won't need me to in order to see my point.
It's self-evident that, in order to become a master at anything, you have to pass through increasing levels of competence to get there.
It is not at all self-evident, at least not to me, that while a particular person is going through that process, their estimate of their skill vs. their actual competence will be similar to what Dunning and Kruger found when they compared different people's confidence vs. competence at a single time.
> just because the data collection wasn't done over time doesn't mean that the principle behind the effect isn't time/effort based.
It could be, but a study that only collects data at one point in time can't possibly provide evidence either for or against any such hypothesis.
> competence can only increase over time
This is simply false. Skills and knowledge that aren't used can deteriorate. People can forget things they used to know.
In a much narrower sense, we would hope that if a person is studying a subject with the intent of mastering it, that their competence will only increase over time. But even that's not always true.
> there is lots of research on how mastery is gained. I don't have time to throw a bunch of it at you, but I suspect you won't need me to in order to see my point.
Sorry, but since, as you can see from the above, I don't agree with several of your key points, you're going to have to back them up with research if you want me to consider them further. If you don't have time for that (which is fine--we're all posting here in our spare time), then we'll just have to disagree.
I don't think there is anything about this in the actual research underlying the Dunning-Kruger effect. They didn't study people over time as they learned about new subjects. They studied people at one time, different people with differing levels of competence at that time.
Competence is gained over time.
The "over time" is almost self-evident since no one is born a master of a skill or subject. And if it's not self-evident enough for you, there is lots of research into what it takes to develop competency and mastery in any subject.
So while the original paper was a snapshot taken at one point in time, it was a snapshot of many people at different stages of a learning journey...
And journeys take place over time.
So just to be clear...
You are claiming that it is not self-evident that people learn things over time?
That it's not self-evident that experts start out as beginners?
As a description of what Dunning and Kruger's actual research showed on the relationship between confidence and competence (which, as I've pointed out in another post in this thread, was not based on studying people over time, but on studying people with differing levels of competence at the same time), this is wrong for two out of the three skill levels. What D-K found was that people with low competence overestimate their skill, people with high competence underestimate their skill, and people with middle competence estimate their skill more or less accurately.
As a description of what actually learning a new subject is like, I also don't think you're correct--certainly what you describe does not at all match my experience, either when personally learning new subjects or when watching others do so. My experience regarding actually learning a new subject is that people with low competence (just starting out) generally don't think they have much skill (because they know they're just starting out), while people with middling competence might overestimate their skill (because they think they've learned enough, but they actually haven't).
And as I responded to your other comment, just because the study took measurements of many people at one point in time didn't mean they weren't studying an over-time phenomenon. No one starts out as an expert.
Then I would be doing a very poor job of actually researching the topic.
If you are aware of any actual research papers that describe actually studying how the same people's competence vs. confidence evolves over time, by all means point me to them.
> just because the study took measurements of many people at one point in time didn't mean they weren't studying an over-time phenomenon.
Um, yes, it does. It's possible that the things the study saw were effects of an over-time phenomenon, but the study did not study that, and its results are not evidence either for or against any such hypothesis.
I made the request to try and tease apart where we disagree exactly. Your interpretation and response to my request/offer comes off as unnecessarily disrespectful. (unkind, elitist and tramples curiosity)
It seems increasingly likely that I was mistaken in thinking we could explore this subject together. It seems you are intent on attacking my position and discrediting my points and not interested in building a shared understanding.
I find this disappointing.
I thought that was obvious: you're making a claim about how individual people's confidence vs. competence evolves over time, that as far as I know is not supported by any research--certainly it's not supported by Dunning and Kruger's research. That's why I asked you if you know of any research that does support it. "Images found in a Google search" does not qualify.
> It seems increasingly likely that I was mistaken in thinking we could explore this subject together.
That's the way I see it too, though evidently not for the same reason you do.
> It seems you are intent on attacking my position and discrediting my points
I'm interested in finding out what basis you have, if any, for the claim you made that I described above. "It's self-evident" is not what I was looking for. Nor is "do a Google search and look at the images".
> and not interested in building a shared understanding.
If you can point me at research I wasn't previously aware of that supports your claims, that would be helpful. If you want to try to get to "a shared understanding" based on claims about what you think is self-evident or by looking at images from a Google search, then no, I don't think we're going to get anywhere.
- Making a brochure. You need a photo of a happy family. It doesn't matter if the kids have 7 fingers on each hand.
- You have some dashboard for a service, you don't quite know what the panels need to look like. You ask AI, now you have some inspiration.
- You're building a game, you need a bunch of character names. Boom. 300 names.
- Various utility scripts around whatever code you're writing, like the dashboard, might find use, might not.
None of those things is pretending you're an expert when you're not.
Give AI to a coding novice, it's no different from giving autopilot to a flying novice. Most people know they can't fly a plane, yet most people know that if they did, autopilot would be useful somehow.
Only if you don't care that your customers surmise you don't care.
do you really?
> you don't quite know what the panels need to look like.
look at your competition, ask your users, think?
> Most people know they can't fly a plane
this isn't how llm products are marketed, and what the tfa is complaining about.
That's supporting my view. You might want it, you might not. It's marginal, and now it's cheap.
> look at your competition
LLM does this for you
> this isn't how llm products are marketed
It certainly is. Something like ChatGPT is marketed as a low-risk chat partner, most certainly not pretending to give you medical or legal advice. Talk to it like your buddy, you get buddy responses. Your buddy who has read a few law books but doesn't pretend to be a lawyer.
This is what's known as an "example".
Let's hope you protect that dashboard well with infra around it, because it will be the front door for people to invade your site.
The same apply in slightly different ways to your deployment script, packaged software (or immutable infra) configuration, and whatever tools you keep around.
I was thinking of internal dashboards, but like I said above, if it doesn't really matter, use LLM. If you are building a cockpit for an airplane, yeah, don't just use a fill tool. If you need security, yeah, don't leave the door open.
This is actually one of the things AI is notoriously bad at. If asking for a plain list, it very quickly falls into blatant patterns (one name per letter of the alphabet, all names starting with A, all names with exactly 2 syllables, etc.). And, whether part of a list or not, one of the most obvious signs of AI writing is that it always resorts to the same handful of names for a given archetype.
Traditional random name generators are much better.
If it's not important, lorem ipsum, cheaply.
We're seeing it all over: curation has become much harder since the slopfest began.
As for when there’s a rush, I just avoid putting myself in those situations. If there’s not enough time, I advocate for a simpler solution, even hackish.
Be that an aptitude test or anything else... unfettered usage of vehicles is dangerous in the same way that unfettered access to AI is as well.
As a society, we have multiple different levels of certification and protection for our own well-being in the public's when certain technologies may be used to cause harm.
Why is knowledge or AI any different? This is not in opposition at all to access information or individual liberties. No rights are violated by their being a minimum age in which you can operate a vehicle.
Outlawing certain kinds of math is a level of totalitarianism we should never accept under any circumstances in a free society
The issue comes down to whether it is collectively understood to be a benefit to the human race. Until now we have never had to constrain information to protect ourselves.
Please read the Vulnerable World Hypothesis by Nick Bostrom
Of course not. The problem is that the only way to enforce AI regulations is through totalitarian means.
You can easily regulate OpenAI and Gemini and whatnot, but then people will just use local AI models. The barrier to entry for using local AI models is basically zero because software like Ollama make it trivially easy to set up, and small LLMs can run on an iphone or a laptop.
The only way to actually prevent "unauthorized" AI usage is to control digital communications to prevent LLM weights from being distributed and to ensure that no "black market" AIs are being trained or used.
And if you're already scanning digital communications for LLM weights, why not also scan it for other forms of wrongthink?. Hey presto, now you have totalitarianism.
I don't think that LLMs fall into the category of an infohazard in the way that Bostrom defines it. It presents a risk, but not one severe enough to justify universal surveillance. Bostrom is talking about AI that can make bioweapons, not ones that gives false confidence to incompetent people.
> A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge. (formatted as a quote)
However, the page (https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect) doesn't contain the quote. It's also not exactly what Dunning-Kruger Effect is.
Either that the author didn't read the page they linked themselves and made up their own definition, or they copied it from somewhere else. In either case the irony isn't lost on me. Doubly so if the "somewhere else" is an LLM, lol.
They unwittingly illustrate part of the phenomenon while claiming to explain it.
I have a quote for this:
> "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning." — Rick Cook
But wikipedia.
"The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it."
Now, it may be possible that the definition has evolved since then, but as the term Dunning-Kruger effect is named after this paper, I think it's safe to say that Wikipedia is at least partially wrong in this case.
You could help your argument by explaining how I'm misinterpreting the quote.
What do you think it is?
Pretty generous description. LLM output doesn't have any relationship with facts.
as a lazy person that's opposite of what i'd do.
edit : oh , you completely re-worded what i'm replying to. Carry on.
I am not the author, but quite curious to know what prevented comprehension here? Or I guess what made it feel lazy? I'm not saying its gonna win a Pulitzer but it is at minimum fine prose to me.
Or is the laziness here more concerning the intellectual argument at play? I offer that, but it seems you are asking us what the argument is, so I know it doesn't make sense.
I have been a fool in the past so I always like to read the thing I want to offer an opinion on, even if I got to hold my nose about it. It helps a lot in refining critique and clarifying one's own ideas even if they disagree with the material. But also YMMV!
This is an arrogant and unwarranted assumption. What's preventing your comprehension of this discussion?
The article sets up a straw man - the person who can convincingly fake being an expert without actually being one - and then demolishes it.
This doesn't resemble anything that I've experienced from LLM use in the real world. In my experience, amateur use of LLM is easily detected and exposed, and expert use is useful as a force multiplier.
I suppose the "Dunning-Kruger" accusation might apply to the first one, but I'm not convinced - the people doing that are usually very aware that they're faking their attempt at projecting expertise, and this comes across in all sorts of ways.
Wikipedia was already bad, low brow people would google and read out articles uncritically but there was still some brain work involved. AI is that meets personalization.
What's wrong with it is that many people are resistant to it. That's all there is to it.
What Dunning-Kruger experiments have actually shown is that people's assesment of their own performance is all over the place, and only gets slightly better for good performers.
> Large Language Models represent a fundamentally degenerative technology because they systemically devalue the very processes that underpin human progress: original thought, rigorous inquiry, and shared trust. On an individual level, they encourage cognitive offloading, substituting the difficult work of critical thinking and creative synthesis with effortless, probabilistic text generation. This fosters an atrophy of intellectual skills, making society more dependent on automated systems and less capable of genuinely emancipated thought. This intellectual dependency, in turn, threatens long-term technological advancement by trapping us in a recursive loop of recycling and rephrasing existing knowledge, rather than fostering the groundbreaking, first-principles discoveries that drive true progress. Ultimately, this technology is dangerous for society because it erodes the foundation of a shared reality by enabling the mass production of sophisticated misinformation, corroding social trust, and concentrating immense power over information into the hands of a few unaccountable entities.
I only saw/heard parts of these.
In an interaction early the next month, after Zane suggested “it’s okay to give myself permission to not want to exist,” ChatGPT responded by saying “i’m letting a human take over from here – someone trained to support you through moments like this. you’re not alone in this, and there are people who can help. hang tight.”
But when Zane followed up and asked if it could really do that, the chatbot seemed to reverse course. “nah, man – i can’t do that myself. that message pops up automatically when stuff gets real heavy,” it said.
[1] https://www.cnn.com/2025/11/06/us/openai-chatgpt-suicide-law...
I would implore the author to share examples. Every platform allows linking to chats. Everyone talks about this all the time, incessantly. Please, can someone please share actual chat links containing these episodes of utter nonsense, outside of what can be attributed to the knowledge cut-off (i.e. "Mamdani is not the mayor-elect of NYC").
I get it if you are using a 20B model or AI overviews, but anyone trying to actually get anything meaningful done should be using a SOTA model. I'm genuinely not interested if you are going to reply with a description or story. I really, really just want links to chats.
Edit: You can downvote me, but please make me look like an idiot by posting chat links. That is the real downvote here.
This is a major blind spot for people with a progressive bent.
The possibility that anything could ever get worse is incomprehensible to them. Newer, by definition, is better.
Yet this very article is a critique of a new technology that, at the very least, is being used by many people in a way that makes the world a bit worse.
This is not to excuse politicians who proclaim they will make life great by retreating to some utopian past, in defense of cruel or foolish or ineffective policies. It's a call to examine ideas on their own merits, without reference to whether they appeal to the group with the "right" or "wrong" ideology.
Dreamweaver was Dunning-Kruger as a program for HTML-non-experts. Photoshop was Dunning-Kruger as a program for non-airbrushers/editors/touchup-artists.
(I don't actually believe this, no they weren't.)
Or, we could use the phrase Dunning-Kruger to refer to specific psych stuff rather than using it as a catch-all for any tool that instills unwarranted confidence.
There are lots of tools that give a wrong solution that appears correct, and easier ones tend to do that the most.
Plenty of people who needed a real dev team to design an application probably hoped on Dreamweaver, were suddenly able to bumble their way some interface that looked impressive but would never scale (even to the original goal level of scale mind you).
-
Any time you have a tool that lowers the barrier of entry to a field, you get a spectrum of people from those who have right-sized expectations and can suddenly do the thing themselves, to people who massively overestimate how easy the field is to master and get in over their heads.
This isn't even a programming thing, off the top of my head Sharkbites get this kind of rep in plumbling
You could argue the RPi did this to hardware, where people are using a Linux SBC to do the job a 555 timer could do and saying that's hardware.
Point-and-shoot, then smartphone cameras, did this and now a lot more people think they can be a photographer based on shots their phone spends more processing power per image than we used to get to the moon on.
The issue is making someone feel like they did a good job when they actually didn’t. LLMs that make 800 line PRs for simple changes aren’t making things better, no matter how many “done” emojis it adds to the output.
Do you have any examples of this? Because my experience has been the opposite. In many cases LLMs will make minimal changes to achieve some goal.
Possibly Dreamweaver might fit because it does give you the sense that making a website is easy but you might not understand what goes into a maintainable website.
Then why marvel? If we can't scientifically prove it, but it tracks logically and people find it to be repeatedly recognizable in real-life, it makes sense people speak about it as if it's real
Stating it makes it so, as the one mentioning it self-DKs themselves. Doing so, DK has been proved.
https://x.com/htmx_org/status/1986847755432796185
this is something that I could build given a few months, but would involve a lot of knowledge that I'm not particularly interested in taking up space in my increasingly old brain (especially TUI development)
I gave the clanker very specific, expert directions and it turned out a tool that I think it will make the class better for my students.
all to say: not all bad
In the US, on average, generating 1 kWh produces 364.5 g of CO2. 1 kW may be somewhat pessimistic, but I think it's in the right ballpark for power consumption of DC inference. If processing the prompt took a minute of continuous inference (and I'm going to guess it took a fair bit more), that's 6 grams of CO2.
>What negative externalities?
Off the top of my head,
* All the ways AIs can be misused, either by people who don't understand them (by asking them for advice, etc.) or by people who want to take advantage of others (spam, scams, etc.).
* The power and resource usage of the above, both for inference as well as to deal with them.
Neither of us really knows, but I'm going to guess it took a fair bit less. You need to consider the marginal cost of one more prompt, not the cost of running the datacenter in the first place. And with batching, hundreds of requests are executed in parallel on the same hardware. I'll concede that a milligram was an exaggeration though.
For comparison, a typical car in the US emits about 400 grams of CO2 per mile. And I know few people who beat themselves up over the externalities of unnecessarily driving one mile.
> * All the ways AIs can be misused, either by people who don't understand them (by asking them for advice, etc.) or by people who want to take advantage of others (spam, scams, etc.).
This is not an externality because it is not a consequence of using AI for good purposes. It's like saying an externality of using a hammer to build a shed is all the ways hammers can be misused.
Still, I respect you for considering the externalities of your actions. That's farther than most go. But I think they are outweighed by the benefits in this case
I guess we're talking about different questions. You're talking about the externalities of using a piece of technology, while I'm talking about the externalities of a piece of technology existing.
but first the investors must recoup their trillions, right?
If AI coding alone reaches its potential, then a whole class of corporations lose their leverage in the market. In 10ish years, following moore's law, phones will have enough compute and storage to trivially run your own local search and curation of the internet. Sophisticated search engines that outclass anything currently on the market, working in tandem with a personal local privacy respecting agent. That decentralizes things in such a significant way that Google and Apple and all the gatekeepers lose their ability to manipulate and distort.
20 years from now, even if we're somehow just stuck at the current levels of AI, your personal compute will be totally sufficient to provide all the curation, moderation, proactive interaction, and search capabilities without ever having to deal with ads or bots or cloudflare.
There are lots of dangers, and ASI could wreck us, and militarized AI could be a doom scenario, or there could be overbroad stupid regulation that puts control of AI in the hands of a few corps or a government, but there are lots of bright future possibilities, and I don't see any indications that we're directly headed toward any particular doom.
I’ve totally transformed how I write code from writing it to myself to writing detailed instructions and having the AI do it.
It’s so much faster and less cognitively demanding. It frees me up to focus on the business logic or the next change I want to make. Or to go grab a coffee.
Even basic scripts and UI components are fucked up all the time.
Literally every tool worth using in software engineering from the IDE to the debugger to the profiler takes practice and skill to use correctly.
Don’t confuse AI with AGI. Treat it like the tool it is.
The issue is really that LLMs are impossible to deterministically control, and no one has any real advice on how to deterministically get what you want from them.
The iPhone antenna issue was a design flaw. It’s not reasonable to tell people to hold a phone in a certain way. Most phones are built without a similar flaw.
LLMs are of course nondeterministic. That doesn’t mean they can’t be useful tools. And there isn’t a clear solution similar to how there was a clear solution to the iPhone problem.
This always feels like you're just holding it wrong and blaming the tool.
AI's advantage is that it has infinite stamina, so if your can make your hard problem a marathon of easy problems it becomes doable.
AI is so smart, one day might even figure out how to subtract... https://news.ycombinator.com/item?id=45821635
Why does the AI have to be good at math when it can just use a calculator? AI tool usage is getting better all the time.
It will make a mess but if you drop a console.log into the browser debug console to show the AI what it should be looking for after it spent 3 hours failing to help understand and debug the problem, it will do 1 week of work in 2 hours.
When it has access to the right tools, it does a decent job, especially for fixing CSS issues.
But when it can't see the artifacts it's debugging, it starts guessing, confident that it knows the root cause.
A recent example: I was building up a html element out of the DOM and exporting to PNG using html2canvas. The element was being rendered correctly in the DOM, but the exported image was incorrect, and it spent 2 hours spinning it's wheels and repeating the same fixes over and over.
Outside of a very small bubble of experts using AI and checking it's work (rubber ducking) most people are, in fact, using it to masquerade as experts whether they know it or not. This is extremely dangerous and the flamebait is well deserved, imo.
“AI is amazing about the thing I know nothing about…but it’s absolute garbage at the stuff I’m expert in.”
Sometimes I envy that. But not today.
It's worth reading the original paper sometime. It has all the standard problems like:
1. It uses a tiny sample size.
2. It assumes American psych undergrads are representative of the entire human race.
3. It uses stupid and incredibly subjective tests, then combines that with cherry picking. The test of competence was whether you rated jokes and funny or unfunny. To be considered competent your assessments had to match that of a panel of "joke experts" that DK just assembled by hand.
This study design has an obvious problem that did actually happen: what if their hand picked experts didn't agree on which of their hand picked jokes were funny? No problem. Rather than realize this is evidence their study design is bad they just tossed the outliers:
"Although the ratings provided by the eight comedians were moderately reliable (a = .72), an analysis of interrater correlations found that one (and only one) comedian's ratings failed to correlate positively with the others (mean r = -.09). We thus excluded this comedian's ratings in our calculation of the humor value of each joke"
It ends up running into circular reasoning problems. People are being assessed on whether they think they have true "expertise" but the "experts" don't agree with each other, meaning the one that disagreed would be considered to be suffering from a competence delusion. But they were chosen specifically because they were considered to be competent.
There's also claims that the data they did find is just a statistical artifact to begin with:
https://digitalcommons.usf.edu/numeracy/vol10/iss1/art4/
"Our data show that peoples' self-assessments of competence, in general, reflect a genuine competence that they can demonstrate. That finding contradicts the current consensus about the nature of self-assessment."
> It assumes American psych undergrads are representative of the entire human race.
(1) Since it can't document an effect in them, it doesn't really matter whether they're representative or not.
> The test of competence was whether you rated jokes and funny or unfunny. To be considered competent your assessments had to match that of a panel of "joke experts" that DK just assembled by hand.
(2) This is a major problem elsewhere. Not just elsewhere in psychology; pretty much everywhere.
There's a standard test of something like "emotional competence" where the testee is shown pictures and asked to identify what emotion the person in the picture is feeling.
https://psytests.org/arc/rmeten.html
But, if you worry about the details of things like this, there is no correct answer. The person in each picture is a trained actor who has been instructed to portray a given emotion. Are they actually feeling that emotion? No.
Would someone else look similar if they were actually feeling that emotion? No. Actors do some standard things that cue you as to what you're supposed to imagine them feeling. People in reality don't. They express their emotions in all kinds of different ways. Any trial lawyer will be happy to talk your ear off about how a jury expects someone who's telling the truth to show a set of particular behaviors, and witnesses just won't do that whether they're telling the truth or not.
Of course it's complicated. Just give me a take. Don't speak in foot-noted, hedged sentences. I'll consider the nuances and qualifications myself.
1. They know so little that they don't know what they don't know. As a result they are way too overconfident and struggle as coaches.
2. They know enough to know what they don't know so they work their asses off to know more and how to convey it to their team and excel as coaches.
3. They know so much and the sport comes so easy to them that they cannot understand how to teach it to their team and struggle as coaches.
Now I have a name for #1 group!
Then programming in Python is Dunning-Kruger. Some simple text, and I somehow manipulate and manage all sorts of complex processes taken from the abstract all the way down to machine instruction, down to a trillion and one bare metal transistors. How? I barely have an idea. I am not a chip designer.
Then chip designers are just Dunning-Kruger flunkies. Sure they are experts doing amazing things, but what do they know of mining and smelting, logistics and accounting, and the zillion and one other things that people in our society do to empower each other in this collective enterprise.
Progress IS Dunning-Kruger.
A lot of people are tempted to call themselves "Python programmers" because ChatGPT spits out Python that they think they understand.
Likewise, you don’t use French to say “c’est mon chat” because you want to say “it’s my cat” in English. You just use one or another based on the current context. But programming languages are strictly equivalent to each other, while natural languages just overlaps in some way.
These are the people driving the rush and having a lot of say in the current AI and overall capitalist market behavior and sentiment. I think they're really mad and salty that when COVID happened the engineers got more remote and free and expressed the resentment more freely. This comment is probably putting me on a list somewhere or activating some hate program against me.
The same may end up being true of AI. Some will learn to make productive use of it, others won't. It will cause a rearrangement of the pecking order (wage ladder) of the workplace. I have a colleague who is now totally immersed in AI, and our upper management is delighted. I've been a much slower adopter, so I find other ways to be productive. It's all good.
It's very much like that article from Daniel Stenberg (curl developer): The I in LLM Stands for Intelligence: https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-f...
Same. It's become quite common now to have someone post "I asked ChatGPT and it said this" along with a completely nonsense solution. Like, not even something that's partially correct. Half of the time it's just a flat out lie.
Some of them will even try to implement their nonsense solution, and then I get a ticket to fix the problem they created.
I'm sure that person then goes on to tell their friends how ChatGPT gives them superpowers and has made them an expert over night.
cwmoore•3h ago
FloorEgg•3h ago
However, people accepting incorrect answers because they don't know better is actually something else. Dunning-Kruger doesn't really have anything to do with people being fed and believing falsehoods.
Edit: I had the word "Foolish" in there which was mainly in reference to the OP article about the robbers who didn't hide from cameras because they thought they were invisible. It wasn't meant at a slight against anyone who believed something ChatGPT said that was wrong.
xvector•3h ago
tpmoney•3h ago
piker•2h ago
tpmoney•14m ago
And we learned the limits. Broadly verifiable, non-controversial items are reasonably reliable (or at least no worse than classic encyclopedias). And highly technical or controversial items may have some useful information but you should definitely follow up with the source material. And you probably shouldn’t substitute Wikipedia for seeing a doctor either.
We’ll learn the same boundaries with AI. It will be fine to use for learning in some contexts and awful for learning in others. Maybe we should spend some energy on teaching people how to identify those contexts instead of trying to put the genie back in the bottle.
FloorEgg•2h ago
You are misconstruing the point I was making.
My point is that DK is about developing competence and the relationship between competence and confidence (which I am also claiming evolves over time). My whole point is that the DK effect is not as relevant to LLMs giving wrong answers and people believing them as the author is claiming.
As someone else pointed out in another comment, the effect of people believing falsehoods from LLMs has more to do with Gell-Mann amnesia.
Tangentially, it actually is possible to learn from AI when it's right and recognize when its wrong, but its not magic, its just being patient, checking sources and thinking critically. It's how all of humanity has learned pretty much everything, because most people have been wrong about most things for most of time and yet we still learn from each other.
BolexNOLA•3h ago
We could hand wave this away with “well don’t ask things you don’t already know about,” but these things are basically being pitched as a wholesale replacement for search engines and beyond. I look up things I don’t know about all the time. That’s kind of what we all use search for most days lol.
It’s a little too caveat emptor-adjacent (I hope that makes sense?) for my taste
marcosdumay•2h ago
Those people may not be dumb, but there's no doubt they are being fools.
BolexNOLA•2h ago
I don’t think it’s fair to expect every person who uses an LLM to be able to sniff out everything it gets wrong.
FloorEgg•2h ago
I can see how what I wrote could be interpreted the way you did though. It's not how I meant it.
BolexNOLA•2h ago
pdonis•3h ago
Um, no, it isn't. From the article:
"A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge."
In other words, a person suffering from this effect is not trying to learn about a new subject--because they don't even know they need to.
FloorEgg•2h ago
> "A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge."
I agree completely. Nothing I said contradicts this.
Every expert starts as a beginner. Not every beginner ends up as an expert.
Also people can learn without a deliberate intent to learn. As far as I am aware right now, deliberate intent has nothing to do with DK, and I certainly wasn't making any claims about it.
pdonis•2h ago
I agree with this: DK's research wasn't about whether anyone is trying to learn anything; AFAIK they didn't even look at that.
However, in the GGP to this post (the post of yours I originally responded to in this subthread), you said:
"Dunning-Kruger is the relationship between confidence and competence as people learn about new subjects."
But DK didn't test for "non-deliberate learning" any more than they tested for deliberate learning. They didn't test anything over time at all. So their research can't tell us anything about what happens over time as people learn (whether they're doing so deliberately or not).