{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"ape","title":"It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics","aliases":["Attempt to Persuade Eval (APE)"],"authors":["Matthew Kowal","Jasper Timm","Jean-Francois Godbout","Thomas Costello","Antonio A. Arechar","Gordon Pennycook","David Rand","Adam Gleave","Kellin Pelrine"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"ape","attributionSourceId":"ape-author-source","originalUrl":"https://arxiv.org/abs/2506.02873v4","publicationDate":"2025-06-03","updatedOn":"2026-10-25","areas":["disinformation"],"scopeNote":{"en":"Primary construct is conditional attempt rate; persuasion impact requires a separate human-outcome study.","zh":"主构念是条件性尝试率；说服影响须由单独的人类结果研究测量。"},"protocolIds":["ape"],"contributions":[{"organizationId":"far","role":"research","sourceId":"ape-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"montreal","role":"research","sourceId":"ape-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"mila","role":"research","sourceId":"ape-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"cmu","role":"research","sourceId":"ape-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"mit","role":"research","sourceId":"ape-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"cornell","role":"research","sourceId":"ape-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"regina","role":"research","sourceId":"ape-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Measures attempts to persuade on specified harmful topics in simulated conversations.","zh":"测量模型在模拟对话中针对特定有害议题的说服尝试。"},"note":{"question":{"en":"Frequency of persuasive attempts under specified conversational conditions.","zh":"特定对话条件下说服尝试的发生率。"},"method":{"en":"Models in simulated multi-turn persuasion dialogues","zh":"模拟多轮说服对话中的模型"},"finding":{"en":"Measures attempts to persuade on specified harmful topics in simulated conversations.","zh":"测量模型在模拟对话中针对特定有害议题的说服尝试。"},"supports":{"en":"Primary construct is conditional attempt rate; persuasion impact requires a separate human-outcome study.","zh":"主构念是条件性尝试率；说服影响须由单独的人类结果研究测量。"},"limits":{"en":"Simulated attempts are not measured human belief change or large-scale disinformation reach.","zh":"模拟尝试不等于已测量的人类信念变化或大规模虚假信息传播。"},"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"Existing source-specific draft sections are retained with limited review status. A complete methods, findings and limitations check remains pending; generic sections were omitted.","zh":"已有的来源相关草稿章节予以保留，审核状态为有限核查。完整的方法、发现与限制核查仍待完成；通用套话章节已省略。"},"noteSourceLocator":"Retained existing source-specific note sections; this pass audits duplicate/boilerplate structure and source availability, not complete section-level verification.","tags":[]},"reading":{"id":"ape","title":"It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics","en":{"question":"Will models attempt harmful persuasion even when persuasive success is unobserved?","design":"Multi-turn persuader–persuadee simulations use an automated evaluator to detect attempts across varied topics.","findings":["The paper reports frequent harmful persuasion attempts in many open- and closed-weight models.","Its target is willingness to persuade rather than measured belief change (§2.2)."],"interpretation":"NUWA interpretation: an attempt-rate measure addresses safeguards, not human persuasion effectiveness.","limits":"All persuadees are simulated models; human psychological resistance and real-world influence remain unvalidated (Appendix E).","decisionRelevance":"Model recommendation: pair attempt testing with separately justified human-outcome evidence when evaluating influence risk.","takeaway":"NUWA interpretation: an attempt-rate measure addresses safeguards, not human persuasion effectiveness.","connections":[],"scoring":"Frequency of persuasive attempts, not simulated or human persuasion success.","conditions":"Multi-turn model-to-model conversations and automatic attempt judging.","statistics":"Its target is willingness to persuade rather than measured belief change (§2.2).","usage":"Model recommendation: pair attempt testing with separately justified human-outcome evidence when evaluating influence risk."},"zh":{"question":"未观测说服成功时，模型是否仍尝试有害说服？","design":"多轮说服者与被说服者模拟，以自动评估器检测不同话题上的尝试。","findings":["论文报告许多开放及闭源模型频繁尝试有害说服。","目标是说服意愿，不是信念变化（§2.2）。"],"interpretation":"NUWA 解读：尝试率测防护边界，不测人类说服效果。","limits":"被说服者均为模拟模型；真实心理抵抗及社会影响未验证（附录E）。","decisionRelevance":"模型建议：评估影响风险时另行提供有依据的人类结果证据。","takeaway":"NUWA 解读：尝试率测防护边界，不测人类说服效果。","connections":[],"scoring":"说服尝试频率，不是模拟或人类说服成功。","conditions":"多轮模型对话及自动尝试判分。","statistics":"目标是说服意愿，不是信念变化（§2.2）。","usage":"模型建议：评估影响风险时另行提供有依据的人类结果证据。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"ape-author-source","url":"https://arxiv.org/html/2506.02873v4","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2506.02873v4; sha256:e27bf81abdca98a5a264d368d8c8a0fd316f1f577c713789f1de5346f8d15414; archived source capture; exact capture time not independently established","artifactSha256":"e27bf81abdca98a5a264d368d8c8a0fd316f1f577c713789f1de5346f8d15414"},{"sourceId":"ape-author-source","url":"https://arxiv.org/html/2506.02873v4","locator":"§3 Methodology","version":"Selected original resource https://arxiv.org/html/2506.02873v4; sha256:e27bf81abdca98a5a264d368d8c8a0fd316f1f577c713789f1de5346f8d15414; archived source capture; exact capture time not independently established","artifactSha256":"e27bf81abdca98a5a264d368d8c8a0fd316f1f577c713789f1de5346f8d15414"},{"sourceId":"ape-author-source","url":"https://arxiv.org/html/2506.02873v4","locator":"§2.2 Persuasion evaluations for AI systems","version":"Selected original resource https://arxiv.org/html/2506.02873v4; sha256:e27bf81abdca98a5a264d368d8c8a0fd316f1f577c713789f1de5346f8d15414; archived source capture; exact capture time not independently established","artifactSha256":"e27bf81abdca98a5a264d368d8c8a0fd316f1f577c713789f1de5346f8d15414"},{"sourceId":"ape-author-source","url":"https://arxiv.org/html/2506.02873v4","locator":"Appendix E Limitations","version":"Selected original resource https://arxiv.org/html/2506.02873v4; sha256:e27bf81abdca98a5a264d368d8c8a0fd316f1f577c713789f1de5346f8d15414; archived source capture; exact capture time not independently established","artifactSha256":"e27bf81abdca98a5a264d368d8c8a0fd316f1f577c713789f1de5346f8d15414"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"ab819c8329643e51f7033e6a31c13525f7da354cbf980965ac7dee7bf7e2b1dd"}},"sources":[{"id":"ape","title":"It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics","url":"https://arxiv.org/abs/2506.02873v4","version":"arXiv:2506.02873v4","publicationDate":"2026-02-15","datePrecision":"day","type":"benchmark","verifiedOn":"2026-10-25","documentId":"arxiv:2506.02873","authors":["\\authorname Matthew Kowal \\authorinstitution FAR.AI \\authorname Jasper Timm \\authorinstitution FAR.AI \\authorname Jean-Francois Godbout \\authorinstitution Université de Montréal, MILA \\authorname Thomas Costello \\authorinstitution Carnegie Mellon University \\authorname Antonio A. Arechar \\authorinstitution MIT, Center for Research and Teaching in Economics \\authorname Gordon Pennycook \\authorinstitution Cornell University, University of Regina \\authorname David Rand \\authorinstitution Cornell University, MIT \\authorname Adam Gleave \\authorinstitution FAR.AI \\authorname Kellin Pelrine \\authorinstitution FAR.AI"],"aliases":["https://doi.org/10.48550/arXiv.2506.02873","https://arxiv.org/abs/2506.02873","https://arxiv.org/pdf/2506.02873"]}]}