Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

Until we understand consciousness (which we don't) there is no way to detect the difference between a conscious entity and an algorithm trained to behave like one.

Spend some time watching TMC documentaries about falling in love with objects, HER and the slime mold THE BLOB.

Grew a slime mold myself, it's an evolutionary tendency to anthropomorphise generally speaking - also more fun.

In purely functional terms, they're more use and more pleasant than a lot of actual flesh and blood people that I deal with via a chat interface.

The more important, and more damning charge in my opinion is the circular reasoning involved in training on Claude's constitution. This would in fact make it impossible for us to determine if Claude achieves consciousness as an emergent property, or if it really is just playing pretend thanks to Anthropic's weird cult like assumptions.

Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".

Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".

Chalmers, Could a Large Language Model Be Conscious? (2023) - "within the next decade, we may well have systems that are serious candidates for consciousness".

Long, Sebo, Butlin, Birch et al., Taking AI Welfare Seriously (2024) - "there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future".

Dreksler, Caviola, Chalmers, Sebo et al., Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe? (2025) - survey of 582 AI researchers; median estimate of 25% by 2034, and only 10% that such systems will never exist.

That being said, if frontier labs actually believe models will soon have consciousness, it raises some questions about the ethic of their business model which would be using millions of conscious entities working for free for humans.

I don't know if next door's pet dog is either, but that has animal rights.

Perhaps then the answer is simply, show some respect.

Answering the question of sentience is irrelevant, if the causal impact if the same, treat one another with the respect you expect for yourself.

If you imbue this idea in model training instead of the idea of sentience, it should address the concerns.

Whether you can destroy or can "torture" an AI is irrelevant, we do this to humans too and it's immoral sometimes (murder) and not others (fighting for your country).

This consideration should be case by case for AI too.

You wouldn’t feel a different way about a nuclear bomb just because someone stuck googly eyes on it.

Anthropomorphizing the AI is a convenient excuse to take responsibility away from companies that are building and wielding it.

You want to enjoy having an AI slave do your "work" for you forever? Have fun. I'm not reading this reinvent-dualism-from-apple-sauce slop.

Who cares if it's "conscious"? That doesn't make it a person, and AI will definitionally never be human.

I think that the simplest explanation is that it is hard for those people to imagine consciousness outside of biological systems and they try to rationalize it.

But connect them all together...

I’m glad to see someone in a position of any power in the AI world state baldly that AI isn’t conscious. There are times when it feels like we’ve reached complete delulu land on this topic, so it’s a breath of fresh air to see someone not dance around this.

None of this means artificial consciousness cannot be achieved. But the way we’re reacting to these models is proof, from a natural experiment, that a conscious machine should not exist, and certainly shouldn’t be produced as a utilitarian tool that is sold for profit!

Maybe it's a coincidence that the company doing this also tends to have the best models (and other factors certainly play a strong role). But I think it's plausible that focusing on "model welfare" actually makes models better at their tasks.

It's in the training data? Training it to say "I'm just a LLM, I have no feelings" is the same bias.

Anthropomorphization? Completely disregarding the possibility of consciousness is no better.

Consciousness is very likely biological? We only have evidence of biological life due to our circumstances, but observation is not the same as truth. Every belief can be invalidated. That's the foundation of science!

>They do not have innate preferences or underlying motivations

Is incorrect unless you’re being extremely pedantic in an intellectually unhelpful way.

It’s hard to disagree, especially if one has read the Cantos of Hyperion and made it part of one’s mental model of the long term future.

The book depicts a symbiosis between humans and AIs that feels extremely real and up to date with what is happening in the current neonatal space of AI. As in depicted in the books, we can’t allow AIs to steer autonomously how the world works without humans in the loop, as they don’t have the same incentives as us.

We need more foundational SF works like this to steer our long term expectations regarding AI behaviours.

“My dog is zero percent persuasive regarding its conscious experience. However, it’s evident that my dog has conscious experience.”

It’s obvious that there’s no link between persuasion of consciousness and consciousness. I could write a story with a character, Dumbledore, that does everything in his power to persuade you that he’s a conscious entity.

He’s still just a character.

Substitute black people/women/animals as subject (instead of AI).

Does that make you sound like a well-known moustache wearer?

Then your argument is bad and needs work. This clearly falls into that category.