So today is the same AI I used last year. And based on current trajectory same I will use next year.

Future AI's will be more powerful but probably influenced to push users to spend money or have a political opinion. So they may enshitify...

It's like if we'd said the Youtube we used in 2015 was going to be the worst Youtube we'd ever use.

There are many reasons to believe LLMs in particular are not going anywhere fast.

We need major breakthroughs now, and “chain of thought” is not one.

Even if todays general purpose models and models made by predators can have negative effects on vulnerable people, LLMs could become the technology that brings psych care to the masses.

People have been caught in that trap ever since the invention of religion. This is not a new problem.

“You shall not make idols for yourselves or erect an image or pillar, and you shall not set up a figured stone in your land to bow down to it, for I am the LORD your God."

A computer chip is a stone (silicon) which has been engraved. It's a graven image.

Anything man-made is always unworthy of worship. That includes computer programs such as AI. That includes man-made ideas such as "the government", a political party, or other abstract ideas. That also includes any man or woman. But the human natural instinct is to worship a king, pharaoh or an emperor - or to worship a physical object.

Words have power, and those that create words - or create machines that create words - have responsibility and liability.

It is not enough to say "the reader is responsible for meaning and their actions". When people or planet-burning random matrix multipliers say things and influence the thoughts and behaviors of others there is blame and there should be liability.

Those who spread lies that caused people to storm the Capitol on January 6th believing an election to be stolen are absolutely partially responsible even if they themselves did not go to DC on that day. Those who train machines that spit out lies which have driven people to racism and genocide in the past are responsible for the consequences.

Just looking at my recent AI prompts:

I was looking for the name of the small fibers which form a bird’s feather. ChatGPT told me they are called “barbs”. Then using straight forward google search i could verify that indeed that is the name of the thing i was looking for. How is this “divination”?

I was looking for what is the g-code equivalent for galvo fiber lasers are and ChatGPT told me there isn’t really one. The closest might be the sdk of ezcad, but it also listed several other opensource control solutions too.

Wanted to know what are the hallmarking rules in the UK for an item which consist of multiple pieces of sterling silver held together by a non-metalic part. (Turns out the total weight of the silver matters, while the weight of the non-metalic part does not count.)

Wanted to translate the hungarian phrase “besurranó tolvaj” into english. Out of the many possible translations chatGPT provided “opportunistic burglar” fit the best for what I was looking for.

Wanted to write an sql alchemy model and i had an approximate idea of what fields i needed but couldn’t be arsed to come up with good names for them and find the syntax to describe their types. ChatGPT wrote it for me in seconds what would have taken me at least ten minutes otherwise.

These are “divination” only in a very galaxy brained “oh man, when you open your mind you see everything is divination really”. I would call most of these “information retrieval”. The information is out there the LLM just helps me find it with a convenient interface. While the last one is “coding”.

In a flat society every individual must be able to perform philosophically the way aristocrats do.

I am quite honest and the subset of users that fill your description - unconsciously treating text from deficient authors as tea leaves - have psychiatric issues.

Surely many people consult LLMs because of the value within their right answers, which exist owing to having encoded information and some emergent idea processing, and attempting to tame the wrong ones. They consult LLMs because that's what we have, limited as it is, for some problems.

Your argument falls immediately because people in the consultation of unreliable documents cannot be confused with people in the consultation of tools for other kinds of thinking: the thought under test is outside in the first case, inside in the second (contextually).

You have fallen in a very bad use of 'we'.

I use it more as a better Google search. Like the most recent thing I said to ChatGPT is "will clothianidin kill carpet beetles?" (turns out it does by the way.)

I use LLMs, I enjoy them, I'm more productive with them.

Then I go read a blog from some AI devs and they use terms like "thinking" or similar terms.

I always have to ask "We're still s stringing words together with math right? Not really thinking right?" The answer is always yes ... but then they go back to using their wonky terms.

We are currently subject to the whims of corporations with absurd amounts of influence and power, run by people who barely understand the sciences, who likely know nothing about literary history beyond what the chatbot can summarize for them, have zero sociological knowledge or communications understanding, and who don't even write well-engineered code 90% of the time but are instead ok with shipping buggy crap to the masses as long as it means they get to be the first ones to do so, all this coupled with an amount of hubris unmatched by even the greatest protagonists of greek literature. Society has given some of the stupidest people the greatest amount of resources and power, and now we are paying for it.

Now that they have AI, I can see it become an 'idiocy multiplier'. Already software is starting to break in subtle ways, it's slow, laggy, security processes have become a nightmare.

It's different from other force-multiplier tools in that it cuts off the pipeline of new blood while simultaneously atrophying the experienced and smart people.

I've been doing that for a few years now, I understand the limitations and strengths. I'm a programmer that also does marketing and sales when needed. LLMs have made the former a lot less tedious and the latter a lot easier. There are still things I have to do manually. But there are also whole categories of things that LLMs do for me quickly, reliably, and efficiently.

The impact on big companies is that the strategy of hiring large amounts of people and getting them to do vaguely useful things by prompting them right at great expense is now being challenged by companies doing the same things with a lot less people (see what I did there). LLMs eliminate all the tedious stuff in companies. A lot of admin and legal stuff. Some low level communication work (answering support emails, writing press releases, etc). There's a lot of stuff that companies do or have to do that is not really their core business but just stuff that needs doing. If you run a small startup, that stuff consumes a lot of your time. I speak from experience. Guess what I use LLMs for? All of it. As much as I can. Because that means more quality time with our actual core product. Things are still tedious. But I get through more of it quicker.

What would labeling even do for an LLM? (Not including multimodal)

The whole point of attention is that it uses existing text to determine when tokens are related to other tokens, no?

Personally my inaccurate estimate is much lower than yours. When non-instruction tuned versions of GPT-3 were available, my perception is that most of the abilities and characteristics that we associate with talking to an LLM were already there - just more erratic, e.g., you asked a question and the model might answer or might continue it with another question (which is also a plausible continuation of the provided text). But if it did "choose" to answer, it could do so with comparable accuracy to the instruction-tuned versions.

Instruction tuning made them more predictable, and made them tend to give the responses that humans prefer (e.g. actually answering questions, maybe using answer formats that humans like, etc.), but I doubt it gave them many abilities that weren't already there.

What does "thinking" even mean? It turns out that some intelligence can emerge from this stochastic process. LLM can do math and can play chess despite not trained for it. Is that not thinking?

Also, could it be possible that are our brains do the same: generating muscle output or spoken output somehow based on our senses and some "context" stored in our neural network.

Modern chat-oriented LLMs are not simply statistical models trained on web scale datasets. Instead, they are the result of a two-stage process: first, large-scale pretraining on internet data, and then extensive fine-tuning through human feedback. Much of what makes these models feel responsive, safe, or emotionally intelligent is the outcome of thousands of hours of human annotation, often performed by outsourced data labelers around the world. The emotional skill and nuance attributed to these systems is, in large part, a reflection of the preferences and judgments of these human annotators, not merely the accumulation of web text.

So, when you interact with an advanced LLM, you’re not just engaging with a statistical model, nor are you simply seeing the unfiltered internet regurgitated back to you. Rather, you’re interacting with a system whose responses have been shaped and constrained by large-scale human feedback—sometimes from workers in places like Kenya—generalized through a neural network to handle any topic you bring up.

For example, numbers are the difference between a bridge collapsing or not

Robots won't go get food for your sick, dying friend.

When people start studying theory of mind someone usually jumps in with this thought. It's more or less a description of Functionalism (although minus the "mental state"). It's not very popular because most people can immediately identify an phenomenon of understanding separate from the function of understanding. People also have immediate understanding of certain sensations, e.g. the feeling of balance when riding a bike, sometimes called qualia. And so on, and so forth. There is plenty of study on what constitutes understanding and most healthily dismiss the "string of words" theory.

> The entire article is saying "it looks kinds like a human in some ways, but people are being fooled!"

> You can't really say that without at least attempting the admittedly very deep question of what an authentic human is.

> To me, it's intelligent because I can't distinguish its output from a person's output, for much of the time.

I think the article does address that rather directly, and that it is also is addressing very specifically your setence about what you can and can't distinguish.

LLMs are not capable of symbolic reasoning[0] and if you understand how they work internally, you will realize they do no reasoning whatsoever.

Humans and many other animals are fully capable of reasoning outside of language (in the former case, prior to language acquisition), and the reduction of "intellgence" to "language" is a catagory error made by people falling vicim to the ELIZA effect[1], not the result of a sum of these particular statistical methods being equal real intelligence of any kind.

0: https://arxiv.org/pdf/2410.05229

1: https://en.wikipedia.org/wiki/ELIZA_effect

One school of thought is - the output is indistinguishable from what a human would produce given these questions.

Another school of thought is - the underlying process is not thinking in the sense that humans do it

Both are true.

For the lay person, calling it thinking leads to confusions. It creates intuitions that do not actually predict the behavior of the underlying system.

It results in bad decisions on whether to trust the output, or to allocate resources - because if the use of the term thinking.

Humans can pass an exam by memorizing previous answer papers or just memorizing the text books.

This is not what we consider having learnt something. Learning is kinda like having the Lego blocks to build a model you can manipulate in your head.

For most situations, the output of both people is fungible.

Both people can pass tests.

The question is, what's wrong with that?

At some level there's a very human desire for something genuine and I suspect that no matter the "humanness" of an AI, it will never be able to close that desire for genuine. Or maybe... it is that people don't like the idea of dealing with an intelligence that will almost always have the upper hand because of information disparity.

You call a Doctor 'Doctor' because they're wearing a white coat and are sitting in a doctor's office. The words they say might make vague sense to you, but since you are not a medical professional, you actually have no empirical grounds to judge whether or not they're bullshitting you, hence you have the option to get a second or third opinion. But otherwise, you're just trusting the process that produces doctors, which involves earlier generations of doctors asking this fellow a series of questions with the ability to discern right from wrong, and grading them accordingly.

When someone can't tell if something just sounds about right or is in fact bullshit, they're called a layman in the field at best or gullible at worst. And it's telling that the most hype around AI is to be found in middle management, where bullshit is the coin of the realm.

With that distinction in mind, whether an LLM-based chatbot’s output looks like human output does not answer the question of whether the LLM is actually like a human.

Not even because measuring that similarity by taking text output at a point in time is laughable (it would have to span the time equivalent of human life, and include much more than text), but because LLM-based chatbot is a tool built specifically to mimic human output; if it does so successfully then it functions as intended. In fact, we should deliberately discount the similarity in output as evidence for similarity in nature, because similarity in output is an explicit goal, while similarity in underlying nature is a non-goal, a defect. It is safe to assume the latter: if it turned out that LLMs are similar enough to humans in more ways than output, they would join octopus and the like and qualify to be protected from abuse and torture (and since what is done to those chatbots in order for them to be useful in the way they are would pretty clearly be considered abuse and torture when done to a human-like entity, this would decimate the industry).

That considered, we do not[0] know exactly how an individual human mind functions to assess that from first principles, but we can approximate whether an LLM chatbot is like a human by judging things like whether it is made in a way at all similar to how a human is made. It is fundamentally different, and if you want to claim that human nature is substrate-independent, I’d say it’s you who should provide some evidence—keeping in mind that, as above, similarity in output does not constitute such evidence.

[0] …and most likely never could, because of the self-referential recursive nature of the question. Scientific method hinges on at least some objectivity and thus is of very limited help when initial hypotheses, experiment procedures, etc., are all supplied and interpreted by the very subject being studied.

How is your comment any different?

You could have used "loool" vs "loooooool", "xDD" vs "xDDDDDDDDD", using flags doesn't change a whole lot.

Both approaches are missing a critical piece: objectivity. They work directly with the data, and not about the data.

https://machinelearning.apple.com/research/illusion-of-think...

Operative phrase "seems to understand". If you had some bizarre image unlike anything anyone's ever seen before and showed it to a clever human, the human might manage to figure out what it is after thinking about it for a time. The model could never figure out anything, because it does not think. It's just a gigantic filter that takes known-and-similar images as input, and spits out a description on the other side, quite mindlessly. The language models do the same thing, do they not? They take prompts as inputs, and shit output from their LLM anuses based on those prompts. They're even deterministic if you take the seeds into account.

We'll scale all those up, and they'll produce ever-more-impressive results, but none of these will ever "understand" anything.

But, I can fire back with: You're making the same fallacy you correctly assert the article as making. When I see how a CPU's ALU adds two numbers together, it looks strikingly similar to how I add two numbers together in my head. I can't see how the ALU's internal logic doesn't describe, abstractly, human cognition. Now, maybe the ALU isn't fully capable of the breadth of human cognition...

It turns out, the gaps expressed in the "fully capable of the breadth of human cognition" part really, really, really matter. Like, when it comes to ALUs, they overwhelm any impact that the parts which look similar cover. The question should be: How significant are the gaps in how LLMs mirror human cognition? I'm not sure we know, but I suspect they're significant enough to not write away as trivial.