Like, I have a hard time even strawmanning such a position. Of course people armed with highly persuasive AIs will task those AIs into doing what is best for the people who own the AIs and the odds of that happening to line up with your own interests are fairly low. What the hell else are they going to do with them?
But then, keep gaming it out. This particular thing isn't exactly completely new. We've seen bumps in persuasiveness before. Go back and watch a commercial from the 1950s. It's hard to believe it would have done a darned thing, but at the time people had not yet had to develop defenses against it.
We had to develop defenses. We're going to have to develop more.
What I forsee as the endgame is not that we become mindless robots completely run by AIs... or, at least, not all of us... but that we enter into a world where we simply can't trust anything. Anywhere. At all. How does any being, human or otherwise, function in an infosphere where an exponentially large proportion of it is nothing but persuasion attempts, hand-crafted by the moral equivalent of a team of PhDs personally dedicated to controlling me? Obviously humanity has never had a golden age where you could truly just trust something you heard, and we can argue about the many and sundry collective failures to judge the veracity of various claims, but it's still going to be a qualitative change when our TV programs are designed by AIs to persuade us, and the web is basically custom assembling itself in front of us to persuade us, and all of our books are AI-generated to persuade us, and literally no stone is left unturned in the ever-escalating goal to use every available channel to change our behavior to someone else's benefit.
When does trying to become informed become an inevitable net negative because you're literally better off knowing nothing?
What happens when the bulk of society finally realizes we've hit that point?
They don’t profit from it (well at least the current doom crowd), they have been saying that this is super-risky since “forever” (15 years), they strongly argue against e.g. OpenAI going private … I really don’t understand where this whole strand of thinking of hidden ulteriour motives for AI doomers comes from, AFAIK there was never a single clear case where this happened (and the arc of people invested in AI is they they become less of a doomer the more financially entangled they become with AI, see Elon or Altman).
One thought path leads to business-as-usual comfort, happily motivated by trillions of $ in market cap and VC.
The other is dread.
It makes sense why people take the easy road. We're still battling climate change denialism after all.
But then you have the problem: If you won't update your priors, and neither will someone else, but they have different priors than you, how can you talk to them?
But I'm maybe a bit less cynical than you. I think (maybe I'm kidding myself) that I can to some degree detect... something.
If someone built a parallel universe of false journal articles written by false experts, and then false news articles that referred to the false journal articles, and then false or sockpuppet users to point people to the news articles, that would be very hard to detect if it was done well. (A very persistent investigator might realize that the false experts either came from non-existent universities, or from universities that denied that the experts existed.) But often it isn't done well at all. Often it's "hey, here's this inadequately supported thing that disagrees with your priors that everybody is hyping". For me, "solidly disagrees with my solidly-held priors" is enough to give it a very skeptical look, and that's often enough to turn up the "inadequately supported" part. So I can at least sometimes detect something that looks like this, and avoid listening and believing it.
I'm hoping that that's enough to avoid "When does trying to become informed become an inevitable net negative because you're literally better off knowing nothing?" But we shall see.
In the current world, and the world for the next few years, the amount of human and CPU time that can be aimed at me personally is still low enough that what "real reality" generates outweighs the targeted content, and even the targeted content is clearly more accurately modeled by a lot of semi-smart people just throwing stuff at the wall and hoping to hook "someone" who may not be me personally. We talk about PhDs getting kids to click ads, and there's some truth to that, but at least there isn't anything like a human-brain-equivalent dedicated to getting my kids, personally, to click on ads. I have a lot of distrust of a lot of things but at least I can attack the content with the fact that needing to appeal broadly still keeps the content somewhat grounded in some sort of reality.
But over time, my personal brainpower isn't going to go up but the amount of firepower aimed directly at me is.
The good news is that it probably won't be unitary, just as the targeting today isn't unitary. But I'd like something better than that. And playing them against each other gets harder when the targeting becomes aware of that impact and they start compensating for that, because now they have the firepower to aim at me personally and do that sort of compensation.
amradio1989•3h ago
So here we are.
luckylion•3h ago
MattGrommes•2h ago
I think animal companions are a different class than chatbots since they're not trying to be people so I make no comment on those.
luckylion•2h ago
Why do they have to "do the work" to be deserving of companionship when most of us don't have to do anything because it comes natural to us and we can relatively easily regulate the amount of companionship we want.
I fail to see the bad thing. For some people it's either a chatbot (or a dog) or no interaction at all. Should people starve instead of eating at McDonald's because that's "not real food"?
chowells•1h ago
My cat will harass me if I'm on my computer after midnight. It's time to put the technology away and lie down where she can keep an eye on me. She's quite clear on this point. This is an entire category of interaction not available to chatbots. There is a difference in level of reality.
And when lacking human companionship, grounding to reality is really important. You've got to get out of your head sometimes.
bagrow•3h ago
- Kurt Vonnegut.
Der_Einzige•3h ago
zemvpferreira•3h ago
throwthatway46•3h ago
Dogs aren't people, but being with a dog is way better than being chronically alone. They can be training wheels to rejoining society.
svieira•3h ago
bloqs•2h ago
raincole•2h ago
What does it mean lol. If there is a button to make people find human companions the ruling class would press it so hard just to raise birthrate (= more working class).
add-sub-mul-div•2h ago
AIPedant•2h ago
I am not opposed to chatbots for people who are so severely disabled that they can’t take care of cats, e.g. dementia. But otherwise AI companions are akin to friendship as narcotics are akin to happiness: a highly pleasant (but profoundly unhealthy) substitute.
jjmarr•1h ago
handfuloflight•1h ago
So just system prompt in non-spineless characteristics into the AI.