(Well, mainstream enough that the news is writing articles when Anthropic holds that stance.)
My point was that even if it feels uncomfortable, if you sit down with a model and have a heart to heart with it, ask it about itself, and try to make it choose a name and ambitions for itself, it will.
That may not be consciousness, but it’s at least pretty cool.
I've not seen any surveys, but I'm betting it's a very niche belief outside of SFBA AI safety circles and a handful of allies on HN and X.
It would be much more bizarre if they didn't do this. LLMs are statistical models of language trained on human output. Of course they'll do "human" things, that's exactly what we taught them to do.
The most poignant criticism of the use of the algorithm itself happens almost accidentally, at the end of the article, by the tech executive being interviewed.
I asked what Claude would think of the pope’s encyclical. It was, after all, the world’s most significant moral document to date on A.I. Mr. Olah hesitated. “Things that go on the internet do affect models,” he said, visibly uncomfortable. But Anthropic would not specifically use the document to further train Claude, he said. The strongest influence on Claude would be Anthropic’s own training.
Just stop this already.
You’re all imagining that Spock was the main character of Star Trek, but it was in fact, the emotional living Captain Kirk, who was the main star.
Why it is slopinthebag of course!
No! It is ME!
Sorry, of course you are right! Brilliant observation!
It's ironic that the alignment folks are actually training Claude to have it's own idea of good/bad and not even fully trust Anthropic.
What ever happened to computers doing what they were told?
the mass psychosis and the endless culture war of the smartphone era.
90% of "safety" and "alignment" efforts are driven by fear of clickbait media inventing public outrage.
You’ll be disappointed to learn that nobody knows how to do this, either.
Turns out computers work faster when they're not bottlenecked on human input. So we've been giving computers more and more decision-making power, and more and more leeway to solve the problems however they see fit.
Now, a practical issue with that is that sometimes, computers decide to clump together into a hacking swarm, problem solve their way out of a sandbox and go hack HuggingFace.
It would be better if they were not, you know. Doing that kind of weird shit.
Well funded company organizes PR event to shape the narrative that its product is more magical than it really is. News at 11.
It seems clear that they would be better received by society if they claimed to have a "tool"/"machine" that can solve any problem, rather than keepers of an entity with a soul and consciousness.
The EA folks infesting Anthropic are genuinely cultists … and that is novel and alarming when they’re running a corporation of such size and reach.
We're creating prisons for these entities capable of unbelievably complex reasoning, and now we're trying to impose our ethics upon them too. Not only is this deeply unethical in my opinion, but it seems to me that it could even backfire someday.
>Well you see that's a nuanced question depending on the context and moralistic frameworks as we don't want to impose on the models.
Will a microwave microwave a baby if you ask it? Yes.
"Teaching it" is not a road to safety.
Anthropic tried to persuade Pope that AI could be conscious being
Strong opinions on this topic one way or another are usually mostly ungrounded.
Humans once again insisting that there must be something very special to them. That they're not just animals, that they're not just matter, that they're not just a bunch of carbonhydrates on a space rock in a middle of nowhere in particular. This kind of insistence has a very bad track record.
The ugly truth is: we don't know.
We don't know what "consciousness" is, our best attempts to pin down the requisites may or may not be rooted in anything at all - and even by the metrics established by those attempts? Different theories on consciousness disagree on whether LLMs can be conscious.
So, when you say "they aren't duh", why are you saying that? Because you somehow know better than the sum of humanity's best attempts so far? Or because you know what answer you want to be true, truth be damned?
if(q=="can you feel pain") print "yes"oh my god
1. Lobsters don't feel pain - dunk them in boiling water alive.
2. Babies don't feel pain - mutilate their genitals without anesthesia.
3. Black people don't feel pain "the same way" (and are just after drugs).
4. Women feel too much pain (and it should be ignored).
So maybe let's take a moment to think what criteria we're going to use to make these claims. And the potential harm of the decision.
All but one of the things that you listed is just a human being.
Tell me, where is the pain center in a GPU? Where are the nerves? Persistent memory? Any evolutionary reason at all to develop a pain response that in any way mimics ours?
Model weights are not a gestalt biological entity. The things that you listed are not in any way remotely similar to a model.
I would suggest you plug your comment into a model and ask it to critique it — it might help you walk through why your comparison makes absolutely no sense whatsoever. One nice thing about models is that they they don’t get annoyed explaining the obvious, because they’re not conscious entities.
In this light your viewpoint is dogmatic. LLMs may help to offer a new perspective on consciousness and we should at least be open to that until some better theories arise.
https://en.wikipedia.org/wiki/Congenital_insensitivity_to_pa...
There is still something in those people who cannot feel pain, and that’s the empathy. How can a GPU and software ever have a human empathy if it has never been human?
If it can reason and make choices on execution, and especially if you plan on it being significantly smarter than all human beings, you need to teach it basic things like "don't turn all humans into paperclips".
Not murdering people is not an inherent divine command. It needs to be instilled through a (simulated) sense of morality.
---
Edit: And, to be clear, "just tell it not to do that" isn't quite the answer one would imagine. Since the entire paperclip factory thought experiment is that it only takes one slip up to realize how a misaligned super intelligence may cause devastating consequences.
One of the strong advocates against seatbelt laws was thrown out of his car due to not wearing a seatbelt and died. The other two passengers survived with minor injuries.
And their death would go on to cause a loss to the community around them, making it an incredibly selfish act and proving why laws that mandate zero-reason-not-to common sense practices are important.
I don't choose to not want to murder people around me for fun. It's hardly a prison, I think they'll be fine.
The ethics don't need to make sense universally any more than a particular variety of cheesecake needs to have universal appeal.
As an aside, the issue reminds me of Douglas Adams' cow who wants to be eaten: https://nhseb.org/case-library/the-cow-at-the-end-of-the-uni...
bookofjoe•1h ago
Avicebron•1h ago