"A true summary, the kind a human makes, requires outside context and reference points. Shortening just reworks the information already in the text."

Then later says...

"LLMs operate in a similar way, trading what we would call intelligence for a vast memory of nearly everything humans have ever written. It’s nearly impossible to grasp how much context this gives them to play with"

So, they can't summarize, because they lack context... but they also have an almost ungraspably large amount of context?

Also I got caught on this one kind of irrelevant point regarding the characterization of the Matrix: I would say Matrix is not just diguised as a story about escaping systems of control, it's quite clearly about oppressive systems in society, with specific reference to gender expression. Lilly Wachowski has explicitly stated that it was supposed to be an allegory for gender transition.

Scott Jenson is one of my favorite authors.

He's really big on integrating an understanding of basic human nature, into design.

How do you know LLMs aren't intelligent, if you can't define what that means?

LLMs give a very strong appearance of intelligence, because humans are super receptive to information provided via our native language. We often have to deal with imperfect speakers and writers, and we must infer context and missing information on our own. We do this so well that we don't know we're doing it. LLMs have perfect grammar and we subtly feel that they are extremely smart because subconsciously we recognize that we don't have to think about anything that's said, it is all syntactically perfect.

So, LLMs sort of trick us into masking their true limitations and believing that they are truly thinking; there are even models that call themselves thinking models, but they don't think, they just predict what the user is going to complain about and say that to themselves as an additional, dynamic prompt on top of the one you actually enter.

LLMs are very good at fooling us into the idea that they know anything at all; they don't. And humans are very bad at being discriminate about the source of the information presented to them if it is presented in a friendly way. The combination of those things is what has resulted in the insanely huge AI hype cycle that we are currently living in the middle of. Nearly everyone is overreacting to what LLMs actually are, and the few of us that believe that we sort of see what's actually happening are ignored for being naysayers, buzz-kills, and luddites. Shunned for not drinking the Kool-Aid.

I see statements like this a lot, and I find them unpersuasive because any meaningful definition of "intelligence" is not offered. What, exactly, is the property that humans (allegedly) have and LLMs (allegedly) lack, that allows one to be deemed "intelligent" and the other not?

I see two possibilities:

1. We define "intelligence" as definitionally unique to humans. For example, maybe intelligence depends on the existence of a human soul, or specific to the physical structure of the human brain. In this case, a machine (perhaps an LLM) could achieve "quacks like a duck" behavioral equality to a human mind, and yet would still be excluded from the definition of "intelligent." This definition is therefore not useful if we're interested in the ability of the machine, which it seems to me we are. LLMs are often dismissed as not "intelligent" because they work by inferring output based on learned input, but that alone cannot be a distinguishing characteristic, because that's how humans work as well.

2. We define "intelligence" in a results-oriented way. This means there must be some specific test or behavioral standard that a machine must meet in order to become intelligent. This has been the default definition for a long time, but the goal posts have shifted. Nevertheless, if you're going to disparage LLMs by calling them unintelligent, you should be able to cite a specific results-oriented failure that distinguishes them from "intelligent" humans. Note that this argument cannot refer to the LLMs' implementation or learning model.

That ain't shortening because none of that was in his post.

Humans seem to get wrapped around these concepts like intelligence consciousness etc. because they seem to be the only thing differentiating us from every other animal when in fact it’s all a mirage.

LLM intelligence is in the spot where it is simultaneously genius-level but also just misses the mark a tiny bit, which really sticks out for those who have been around humans their whole lives.

I feel that, just like more modern CGI, this will slowly fade with certain techniques and you just won't notice it when talking to or interacting with AI.

Just like in his post during the whole Matrix discussion.

> "When I asked for examples, it suggested the Matrix and even gave me the “Summary” and “Shortening” text, which I then used here word for word. "

He switches in AI-written text and I bet you were reading along just the same until he pointed it out.

This is our future now I guess.

As opposed to what grandmasters actually did, which was less look ahead and more pattern matching to strengthen the position.

Now LLMs successfully leverage pattern matching, but interestingly it is still a kind of brute force pattern matching, requiring the statistical absorption of all available texts, far more than a human absorbs in a lifetime.

This enables the LLM to interpolate an answer from the structure of the absorbed texts with reasonable statistical relevance. This is still not quite “what humans do” as it still requires brute force statistical analysis of vast amounts of text to achieve pretty good results. For example training on all available Python sources in github and elsewhere (curated to avoid bad examples) yields pretty good results, not how a human would do it, but statistically likely to be pertinent and correct.

My impression is, there is currently one tendency to "over-anthropomorphize" LLMs and treat them like conscious or even superhuman entities (encouraged by AI tech leaders and AGI/Singularity folks) and another to oversimplify them and view them as literal Markov chains that just got lots of training data.

Maybe those articles could help guarding against both extremes.

[1] https://www.verysane.ai/p/do-we-understand-how-neural-networ...

If you keep redefining real intelligence as the set of things machines can’t do, then it’s always going to be true.

This is right out of Community

I heard from Miguel Nicolelis that language is a filter for the human mind, so you can never build a mind from language. I interpreted this like trying to build an orange from its juice.

The "summary vs shortening" distinction is moving the goalposts. They makes the empirical claim that LLMs fail at summarizing novel PDFs without any actual evidence. For a model trained on a huge chunk of the internet, the line between "reworking existing text" and "drawing on external context" is so blurry it's practically meaningless.

Similarly, can we please retire the ELIZA and Deep Blue analogies? Comparing a modern transformer to a 1960s if-then script or a brute-force chess engine is a category error. It's a rhetorical trick to make LLMs seem less novel than they actually are.

And blaming everything on anthropomorphism is an easy out. It lets you dismiss the model's genuinely surprising capabilities by framing it as a simple flaw in human psychology. The interesting question isn't that we anthropomorphize, but why this specific technology is so effective at triggering that response from humans.

The whole piece basically boils down to: "If we define intelligence in a way that is exclusively social and human, then this non-social, non-human thing isn't intelligent." It's a circular argument.

- Even if CEOs are completely out of touch and the tool can't do the job you can still get laid off in an ill informed attempt to replace you. Then when the company doesn't fall over because the leftover people, desperate to keep covering rent fill the gaps it just looks like efficiency to the top.

- I don't think our tendency anthropomorphize LLMs is really the problem here.

They aren’t just intelligence mimics, they are people mimics, and they’re getting better at it with every generation.

Whether they are intelligent or not, whether they are people or not, it ultimately does not matter when it comes to what they can actually do, what they can actually automate. If they mimic a particular scenario or human task well enough that the job gets done, they can replace intelligence even if they are “not intelligent”.

If by now someone still isn’t convinced that LLMs can indeed automate some of those intelligence tasks, then I would argue they are not open to being convinced.

How is that a summary? It reads as a one-liner review I would leave on Letterboxed or something I would say, trying to be pretentious and treating the movie as a work of art. It is a work of art, because all movies are art, but that's an awful summary.

>The original Turing Test was designed to compare two participants chatting through a text-only interface: one AI and one human. The goal was to spot the imposter. Today, the test is simplified from three participants to just two: a human and an LLM.

By the original meaning of the test it's easy to tell an LLM from a human.

In fact, the example provided by the author is a great illustration of this:

> A philosophical exploration of free will and reality disguised as a sci-fi action film about breaking free from systems of control.

The words here refer back to the notions "free will" that is prominent in Western discourse from St. Augustine through Descartes and thereafter and similarly of "sci-fi". These are notions an uneducated East-Asian with limited Internet use and pop culture fluency will simply not understand. They would in fact prefer the latter description. The author and this hypothetical East-Asian live in very different zeitgeists, and correspondingly experience the movie differently and value different summaries of the film. They each prefer a summary that agrees with the zeitgeist, rather than a novel, unorthodox summary (relative to their zeitgeist) that requires a justification as to its deviation.

On the other hand, if you asked LLMs to explain material and concepts, one modality in which it does is use formulaic, novel, and unorthodox analogies to explain it to you. By formulaic and novel, I mean that the objects and scenarios in the analogies are frequently of a certain signature kind that it has been trained with, but it is novel in that the analogies are not found in the wild on the internet.

If you have frequently used LLMs for the purpose of explaining concepts, you will have encountered these analogies and know what I mean by this. The analogies are frequently not too accurate, but they round out the response by giving an ELI5-style answer.

Ironically, the author may have succumbed to LLM sycophancy.

Unless you believe in the magic sauce of a supernatural soul/mind/etc, our brains function as deterministic biological machines. There's essentially nothing that separates the processing and memory potential of silicon from neurons. And if the building blocks can be made analogous, then all the same emergent properties are possible. There's no reason to believe a circuit couldn't be made to behave in exactly the same way as a human brain. I'm not saying that's where LLMs are; only that it is theoretically possible. So if you imagine such a machine, and you deem that it is not intelligent, you have reserved intelligence as an exclusive human trait and this entire discussion is meaningless.

Secondly, although I'm not in either pro or anti LLM-intelligence camp, I find a lot of the arguments against machine intelligence disingenuous and/or unbalanced.

For instance the "Can't process information it's not familiar with" argument. Another commenter stated the case of scientific papers that it doesn't have any reference for, that it may hallucinate a garbage interpretation of the paper. Not surprising, but guess what, a human would do the same thing if they were forced! Imagine holding a gun to someone's head and telling them to explain a concept or system they've never heard of. That's essentially what we're doing with LLMs; obviously we don't need to threaten, because we haven't given them agency to say no.

Another example is the "Can't be novel, unique, or create something completely new." First of all, difficult to prove, but okay let's take it as given that an LLM can't be novel. Can you prove that a human can? We make all these assumptions on how intelligent and creative we are as humans, and how original our thoughts can be... but how original are they, and can we prove it? How do you know your original thought, or Beethoven's 5th, or the fast inverse square root trick was completely separate from any prior influences? Or... was that "original thought" the conglomeration of a thousand smaller inputs and data points that you trained on, that became part of your brains subconscious processing system, and came together in a synthesis that looks like brilliance.

Finally, whenever this discussion comes up with friends I ask them to think of the least intelligent person they know. Then imagine how many of those there are in the world (likely millions). Could you imagine any conceivable test of any length or depth which would designate all the those humans as intelligent, and all the LLMs as not? I certainly can't. I highly doubt there could be anything approaching 100% accuracy at this point.

Ultimately I think we should ditch both the intelligence and consciousness questions. We can't define them in ourselves, we certainly can't define them in another entity. Let's just come to terms with the fact we're highly functioning biological machines who are both scared and excited to have created something so similar to us.

Sure, AI often makes mistakes, but so do humans. Ask a random person a random question and I guarantee that at least 90% of the times an LLM will produce a better answer. So if knowledge or reasoning mistakes are a benchmark, then humans are not intelligent either.

What is the difference then? I feel that whoever claims that AI is not intelligent simply feels uncomfortable thinking that they are, and is going out of their way to rationalize that, by using various double standards. Like, AI making occasional mistakes counts as evidence against it, but the fact that the vast majority of people are incapable of answering some of the most basic questions somehow doesn't count against human intelligence.

And then there are various contrived "definitions of intelligence", such as "living in the physical world". These have nothing to do with cognitive intelligence and are just made up to make humans fit the definition and AI to not fit. Doing "truly novel things"? How many people have ever come up with anything truly novel? What about the other 99.99%, are they intelligent?

I think there's one thing that's tripping people up. Humans do have a very high COLLECTIVE intelligence. Collectively (and given enough time) we're capable of accomplishing great things, much more than current AI. But each individual human on their own is no more intelligent than a caveman (we have essentially the same brain).