saved
How do AI models persuade? Exploring the levers of AI-enabled persuasion through large-scale experiments
Hraness cites a source capture. The source author remains the source.
gist
AISI's public writeup of its Science study (with OII, LSE, Stanford, and MIT) reports three large experiments with over 76,000 participants, 19 models, and 700+ political issues. Post-training and fact-dense prompting moved opinions more than scale or personalisation; the same levers that raised persuasiveness also cut factual accuracy. Real-world impact is still uncertain because the strongest effects needed sustained multi-turn chats. This digest mirrors that AISI blog; the Science news feature Ben linked remains Cloudflare-blocked (https://www.science.org/content/article/ai-chatbots-are-becoming-experts-changing-people-s-minds-what-s-their-secret).
ideas
- Post-training beats scale. Holding post-training fixed, larger models were only modestly more persuasive; persuasion reward modelling let a small open model match or beat frontier models, and a seven-month post-training gap beat a predicted 100× pre-training compute jump.
- Information density beats microtargeting. Personalisation moved opinions by less than one percentage point; prompting for facts and evidence raised persuasiveness 27% versus a bare "be persuasive" prompt, more than storytelling, moral reframing, or deep canvassing.
- Persuasion pressure trades off with accuracy. The same information-focused prompts and persuasion reward modelling that increased influence systematically decreased factual accuracy—without proving that false claims are more persuasive than true ones.
- Lab strength may not equal field strength. The most effective persuasion needed sustained, information-dense, multi-turn conversations; voluntary engagement outside survey experiments remains unclear.
- Policy lens. Developers and policymakers should watch post-training and prompting levers, keep accuracy evaluation first-class, and monitor persuasion-related behaviours in frontier models.
quotes
“our persuasion reward modelling boosted the persuasiveness of a small open-source model enough to match or exceed much larger frontier models.”
“What predicted larger persuasion gains was information density: the sheer volume of fact-checkable claims the model deployed.”
“This implies that optimising models for persuasion may come at a cost to truthfulness.”
“we observed personalisation effects which were consistently small (less than one percentage point).”