However, I'm still skeptical of the effects or size of the change. First, a point that applies to the Massachusetts ballot on psychedelics in particular, putting views into percentages, and getting accurate results from political polls are notoriously difficult tasks [2]. Therefore, the size of any effect is faced with whatever confounding variables make those tasks difficult.
Second, there could be some level Hawthorne effect [3] at play here, such that participants may report being (more) convinced because that's what (they think) is expected of them. I'm not familiar with the recruiting platforms they used, but if they're specialized in paid or otherwise professional surveys, I wonder if participants feel an obligation to perform well.
Third, and somewhat related to the above, participants could state they'd vote Y after initially reporting X preference, because they know it's a low-cost no-commitment claim. In other words, they can claim they'd now vote for Y without fear of judgement because it's a lab environment and an anonymous activity, but they can always go back to their original position once the actual vote happens. To show the size of the effect with respect to other things, researchers will have to make the stakes higher, or follow-up with participants after the vote and find out if/why they changed their mind (again).
Fourth, if one 6-minute-average conversation with a chatbot could convince an average voter, I wonder how much did they know about the issue/candidate being voted on. More cynically for the study, there may be much more at play with actual vote preference than a single dialectic presentation of facts. For example: salient events that happen in the period up to the election; emotional connection with the issue/candidate; personal experiences.
Still, this does not make the study flawed for not covering everything. We can learn a lot from this work, and kudos to the authors for publishing it.
[1] https://www.nature.com/articles/s41586-025-09771-9
[2] For example: https://www.brookings.edu/articles/polling-public-opinion-th...
Chatbots can sway political opinions but are 'substantially' inaccurate: study
marojejian•1mo ago
1) the LLMs mostly used factual information to influence people (vs. say emotional or social influence) 2) the fact were mostly accurate
I'm not saying we shouldn't worry. But I expected the results to be worse.
Overall, the interesting finding here is that that political opinions can be changed by new information at all. I'm curious how this effect would compare to comparably informed human discussions. I would not be surprised if the LLMs were more effect for at least two reasons:
1) Cost-efficiency, in terms of the knowledge required, and effort/skill to provide personalized arguments. 2) Reduction in the emotional barrier to changing your mind: people don't want to "lose" by being wrong about politics to someone else. But perhaps the machine doesn't trigger this social/tribal response.
Cited papers:
https://www.nature.com/articles/s41586-025-09771-9
https://www.science.org/doi/10.1126/science.aea3884
techblueberry•1mo ago
I find this online as well, like I hate being “out of my echo chamber” because those arguments are just uniformly pointless. (This is in all directions by the way, people to the right or left of me).
Though I also interestingly find trying to talk to LLMs about competing values challenging too, if I try to get the LLM to explain a conservative position, then I make counter-arguments to that position, it will almost never tell me my counter argument is wrong, just “you’ve hit the nail on the head! Boy are you smart!”
dlivingston•1mo ago
He never engaged in political conversation with "here's what I believe, and here's why you should too." His approach was more Socratic; to listen to me talk, and then offer an additional viewpoint or context.
I never got the impression from him that he was trying to convince me of something, or that he thought I was wrong about X/Y/Z, but rather, that we were on an intellectual journey together to identify what the problems actually were and what nuanced solutions might look like.
I still have no idea to this day what his ACTUAL political party is (or if he even has one). I genuinely could not tell you if he was left, right, or center.
bossyTeacher•1mo ago
Did you not asking him about HIS position on different matters? That is how I would do it. Some people won't share their views unless directly asked
apercu•1mo ago
Gasoline is like the least important cost metric in my life.
techblueberry•1mo ago
Roger Scruton in I think this video: https://www.youtube.com/watch?v=1eD9RDTl6tM. Says that basically conservatism in the 80's in the UK was whatever Margaret Thatcher believed. This really I think helped me understand why the conservative transition from Reagan/Bush to Trump went more smoothly than I thought it would among trad conservatives.
paulryanrogers•1mo ago
Tribalism was a key substrate. This often manifested as a near blind loyalty to the party and chosen thought leaders like Bill Graham, Rush Limbaugh, Bill O'Reilly, and now Tucker Carlson. They told us how to interpret events and we repeated the talking points. They gave us the (often contradictory) rules and principles we were to use to view everything in life.
zem•1mo ago
ekjhgkejhgk•1mo ago
No, you see. This is how I used to think when I was a teenager.
Democracy isn't about being factually correct. It's about putting in place rules to make accumulation of power to the point that it can bend the rules themselves, very difficult.
It's not a silver lining that LLMs are persuasive by being mostly accurate, if they're used to increase the power of their owner further.