Turns out even doctors doing medicine aren't immune to persuasive AI — if human-in-the-loop was ever a safeguard, consider it the attack surface now. This failure mode exposed here virtually concerns any human expertise imo — disclaimer, I'm an author of this preprint.

Even if AI brings benefits not captured here, given the core principle of 'first, do no harm,' the risks shown in this study alone mean we must be extremely cautious about the value of AI as a decision-support tool.