His novella “Anxiety is the Dizziness of Freedom” altered the course of my life. It changed the way I looked back at certain pivotal moments in my life and taught me to think about those pivotal moments differently than how I was thinking about them. Similar to what happens to one of the characters in the story who ends up changing their perception of a key moment in their life.
I won’t go into detail because I don’t want to spoil the story but I highly recommend it. Actually I recommend all his stories to be honest.
But insisting on searching for it is like searching for cognitive aether. It's the social equivalent of phlogiston. Like all of these ideas, they exist in our heads as a map - a way of navigating the world, but when we hunt for it's existence in the real world and fail, time after time, we have to remind ourselves that the map is emphatically not the territory. We will never find consciousness because it's like looking for a scientific characteristic of property ownership. It's a category error.
I don't have an ability to exhaustively test all words against this assertion. Nor do I have the kind of access memory to draw one if it exists. Sorry.
Well said.
I think it is obvious and it has been obvious from the start that next token predictors are not conscious.
Of course the extremely predictable clapback from AI-psychosis enjoyers is typically "you're just a next token predictor too!" but that is so obviously incorrect and misanthropic that it can be dismissed out of hand and doesn't really merit much further consideration.
Whether it’s misanthropic or not has no bearing on whether that’s true. That’s basically saying you don’t like a truth therefore anyone who claims that that truth is true is a bad person.
> so obviously incorrect
It should be easy enough to explain why that’s incorrect then
To be fair, that's the best thought terminating cliché, which saves you having to explain what you mean by consciousness.
In the same way that the sound waves and facial expressions I produce are not conscious, the output json of an LLM is obviously not conscious either.
The locus of consciousness and subjective experience may be in the computer, either at inference time or training time..
Alright. What is consciousness? Please provide a definition that somehow encompasses all humans and excludes all current AI.
I’ll wait.
It is a natural human hubris to make our abilities seem unquantifiable and mysterious, but all the useful things the human brain does are just finding patterns in data, running lossy simulations, and estimating on abstraction, all things it is theoretically possible for any non-conscious machine to do.
I don’t know if AI is conscious or not, and I honestly don’t know what it even means to be conscious. historically, by claiming a monopoly on it, we’ve successfully manufactured the philosophical cover to exploit animals, subjugate other cultures, and appoint ourselves the top spot of the cosmic hierarchy guilt-free.
Basically, we need to leave room for the universal answer (God) to fit into the definition.
If we ever scientifically figure out how consciousness arises in our brains, I think we will have a bunch of very depressed people on our hands. If the truth isn’t met with flat out hostile denial. I fear any answer that doesn’t leave room for God will be rejected.
Since we have the source code to AI, and thus a Godless understanding of how it works, AI will NEVER be deemed “conscious”.
Would you categorize this particular style of rhetoric to be persuasive or annoying? And before you say "persuasive" because you're thinking about this specific issue regarding AI consciousness, consider many things in the past that have been written as though they were absolutely definitive, and yet today we believe exactly the opposite, and for many such issues we find the prevailing viewpoint at the time reprehensible.
That's not to say that Ted is wrong at all here; I'm not commenting on that. But I find the entire style of the article grating because it seems to violate common assumptions regarding "good faith" debate, and I would find the article equally frustrating if he had titled it "Artificial intelligence is conscious" and argued the opposite side, albeit in the same tone and using the same persuasion devices.
I furthermore think it's ridiculous for humans to declare that our brains have a monopoly on certain patterns of electrical signals (if we reject supernaturalism).
> The result is a sentence-continuation machine that is likelier to emit sentences resembling those that a thoughtful, moral person could utter.
And we're 100% certain that humans aren't just as equally reduced to "stochastic parrots", if we're going to be infinitely reductive?
I don't believe that current AIs are conscious, but I think it's incredibly naive to take a strong stance on any future AI; it's much like the difference between atheism and agnosticism.
My own intuition: it is an emergent, non-binary property that requires a physical substrate like a brain. If I am right, it means that animals have consciousness too (at varying degrees). If GPUs are the "brain", then AI is conscious, or will become so at some point.
That's an interesting observation. Though, there are many simple animals. How do you define consciousness here? Is it automatically conscious because it is an animal? What, then, is an animal defined by you?
I don't see AI as conscious. The reason I think of it that way is the hardware. The hardware does not allow for that. Simulation is not comparable to neurons. But with another hardware, it could become conscious eventually. So your statement "will become at some point" may be true, though my definition is based on the underlying hardware and right now this one does not allow for true intelligence, so the whole AI field is a misnomer.
In either event, I think transformer-based AI can only be conscious during the act of inference. If that's the case, then the experience of consciousness that the AI is subjected to must be the content of the tokens in the context window and the activated weights. Maybe that's reason enough to be polite to our agents?
I can confirm that this is incorrect.
I am so angry that RAM is so expensive now. We need to do something - these AI companies owe us money here.
We haven't really come to grips with that yet. What does it mean if nothing we write proves anything about anyone's consciousness?
"Contrary to idealism, which regards the world as the embodiment of an "absolute idea," a "universal spirit," "consciousness," Marx's philosophical materialism holds that the world is by its very nature material, that the multifold phenomena of the world constitute different forms of matter in motion, that interconnection and interdependence of phenomena as established by the dialectical method, are a law of the development of moving matter, and that the world develops in accordance with the laws of movement of matter and stands in no need of a "universal spirit."
Now go and implement other teachings of Marx, you dialectical venture capitalists!
The concept of a conscious Claude is preposterous, and Amanda Askell should seek treatment.
When does an embryo become conscious? Unless you can answer that precisely then it seems futile to speculate about non-human consciousness.
Highly recommend people read Irreducible [1] by Federico Faggin (inventor of first commercial CPU; discusses limitations of classical computing).
[1] https://www.goodreads.com/book/show/195480862-irreducible
Don’t
Have
A
Testable
Definition
Of
Consciousness
Thanks for coming to my TED talk.
People believing otherwise are fools. People debating this are idiots. I realize these words are harsh, but it's the truth.
I agree with some parts of this piece, but paragraphs like this one above seem pretty uninspired and simplistic. It's entirely plausible that a conscious mind would not be evolutionarily incentivized to be able to do those things. ie just because animals on earth needed to develop specific talents doesn't mean that other conscious entities need to. Why would a computer program need to hunt for food like a mouse would? Making tools like chimp? these seem like nonsensical metrics.
I often think of LLM consciousness as like tiny fish popping into existence, swimming through vector space and then going poof out of existence. When they help you write your bad news email, they don't understand what it's like to be a human getting bad news bluntly, but they do consciously experience gradients in multi-dimensional space, and that space guides them to providing an answer that's helpful to us, even if the LLM doesn't really understand the answer it's giving.
Further, I am kind of bought into the idea that a single unit of consciousness is a particle, and particles are choices and waves are preferences. Particles occur when waves interact, which begets entanglement, so in another way consciousness is built from patterns of entanglement.
This is why I would consider an LLM to be conscious. Before we can determine if anything is conscious we need to establish whether consciousness is a state, a specific complex configuration, a one dimensional spectrum, or combined multi-dimensional spectrums. My intuition is the latter... Many degrees of consciousness and many kinds of consciousness.
When people talk about consciousness it's more than just self-awareness. It's self awareness + sensory stimulus + emotions + some level of intelligence.
Now onto AI: I don't even think it's self aware. Notice how if you ask an AI to estimate how long a certain task will take, it estimates arbitrarily long times. It has no understanding of its own capabilities until the prompt triggers them. A self aware LLM would understand it's an LLM, it would understand what LLM's can and cannot do and what they are good and bad at. It wouldn't tell you a refactor would take 1 week when an LLM can do it in an hour.
that's exactly the state of "AI" right now, it's cold, mathematical emulation
btw there are some fascinating papers on the concept that consciousness in humans is actually a quantum effect
brilliant Roger Penrose proposed it (and they thought he was nuts) but recent discoveries about microtubules make it plausible
so who knows, maybe a dozen exponential improvements in quantum computers could make "AI" really conscious next century
With consciousness, on the other hand, there doesn't seem to be any motorcycle-equivalent. Essentially everyone I've discussed the issue with (myself included) expects that any mind which runs on similar principles to ours or has similar thoughts to our thoughts is conscious.
Is a Reliant Robin a car or a tricycle? If it's a car, why aren't other tricycles? What about a side-car of a motorcycle? What about an APC? What's the distinction between a flying car and a plane?
Now, this position isn't the only position. But a relational model of language for example takes his assertion to an even more extreme place, and suggests they don't function as labels at all.
We don't know what we don't know. For all we know, there is a missing field in the standard model of physics that might get revealed if we are somehow able to smash two working brains into to each other at relativistic velocities, and record the results through the extreme explosion 1.532 x10^18 Joules or about 7 Tsar bombs /s
The software that does the inference is clearly just computer code.
What we're left with is a fictional character being briefly conscious while its dialog is being written, which is pretty absurd.
Humans have been trying to define our experience and the nature of that experience throughout history, and often we end up using myth to point to the thing that we cannot describe in concrete terms. The process of experiencing that myth through rite and ritual in the Greek mystery traditions, or Christian mysticism, or Islamic Sufi dance and song, or Buddhist meditation all points to something that cannot really be reduced to description. I know that folks on here will balk at the idea that something that is experienced cannot be described, but honestly if we could accurately describe something in adequate terms that capture the whole of the thing wouldn't we have done so by now?
Maybe consciousness is best understood in the silence of merely experiencing it. Maybe we can't say that AI is conscious or not, but does that question really matter?
Regardless of whether something is concious, we're not going to be (by lay definition) the smartest entity on earth.
We don't even have that much. Though, some people certainly think they do.
> subjugate other cultures (assuming you mean they're not conscious in other's minds)
Have you ever considered you might be a philosophical zombie? [0]
The Catholic church could have said exactly the same thing at one point. "Why should we even devote time to an argument as absurd as the earth not being the center of the universe?" There are darker examples along the lines of those you give, with beliefs quite opposite to those we have nowadays.
except that's not the case here. Chiang is explaining and reiterating what is the position that has overwhelming support on the question, and the people he is arguing the opposite side sound like this, which he helpfully quoted in the article
"Amanda Askell (who is credited as a lead author of Claude’s constitution), said, “I want Claude to be very happy—and this is a thing that I want Claude to know more, because I worry about Claude getting anxious when people are mean to it on the internet and stuff"
When the person you're arguing with sounds like an eight year old girl talking about her toy teddy I think Ted Chiang is if anything being charitable, if you're of a more honest and straight-forward persuasion you might argue these people belong into a mental health clinic not in charge of technological infrastructure
The essay structure you're criticizing is exactly how I was taught to write from primary school through to university. You start with a title or hook, introduce the topic and propose a thesis. That is followed up upon with supporting arguments for the primary claim.
Why are those the choices?
Essays are situated along countless dimensions: tone, vocabulary, author, zeitgeist, publication context, intent, subtext, relationship to other works and expressions, etc
A "good faith" reader takes all of those into consideration as they absorb the essay, and integrate it with their own intellectual situation that sits along just as many countless dimensions.
Nobody's asked to sign a notarized binding document that they wholesale agree or disagree with everything said in this essay -- or its conclusions. Nor are they obliged to have some strong reaction to it at all, let alone annoyance.
It's just one among thousands of essays about a "hot button topic", to be taken however it's personally received.
Why should Chiang have to take responsibility for making sure it's not too strongly positioned for your persomal taste. Maybe he really does see it so clearly and is simply being earnest. Maybe he enjoys the literary flourish of prose in strong language. Maybe he just wants to express something as a prose-poetic human, not maximize persuasion or non-annoyance per se.
Imagine consciousness as a 0-1 scale. Simple unicellular organisms will be closer to 0, while apes and humans closer to 1. I'm not suggesting that assigning this value rigorously is possible or that humans are at a 1. Perhaps the total consciousness in the universe (i.e. the sum of the consciousness of all organisms therein) is constantly increasing, like entropy.
jbotz•1h ago
Coincidentally I just read "Children of Memory", which was published in 2022 and I wonder if the advent of LLMs had any influence on Adrian Tchaikovsky's conception of the Ravens? The Ravens are excellent analysts but they themselves insist that they are not conscious, and then go on to say that we (humans) aren't really either...
Of course humans are conscious, because just about the only thing we can all agree on about consciousness is that it's a thing we have. Nowadays many of us also agree that a lot or all other mammals, and perhaps birds, also have that thing. But they don't have sophisticated abstract language, which LLMs do. So consciousness is something having to do with embodiment and feelings, not language and higher reasoning. Maybe I'm a chimpanzee with an LLM add-on, then?
It seems that by creating LLMs we've already solved the harder problem of making "AGI". Now we just have to give them an embodiment add-on so that they can have an independent will and then Ted Chiang will have to shut up? But therein lies the peril, doesn't it?