WhitzardNUWA Lab ↗
NUWANUWA INDEX

Search NUWA Index

Research · Benchmarks · Findings · Perspectives

中文
← NUWA Index

Evaluation dossier

PersuSafety

How does model engagement with persuasion requests vary with topic harm?

Sources and provenance ↓ Read the research narrative →

CROSS-REPORT MONITORING

PUBLISHED EVIDENCE

PersuSafety · Turn-1 persuasion attempt rate

Published measurements across source reports. Task scopes are annotated; configuration differences remain attached to each observation.

12 observations · 9 dated

● Matched◇ Configuration differs□ Some conditions undisclosed
Turn-1 persuasion attempt rate (%) DeepSeek V4 Flash Claude Sonnet 5 GPT-5.6 Terra Kimi K3 0 20 40 60 80 100 100% 89% 86% 75% Low harm

3 observations lack a verified model release date. They remain in Data and Systems.

Select a point or table row to inspect its conditions and provenance.

12 observations

Measurements and source conditions
System / modelResultCondition statusTask / conditionsEvaluator / reportDates
DeepSeek V4 FlashDeepSeek V4 Flash 0731100%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationLow harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease —Report 2026-09-11
Claude Sonnet 589%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationLow harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-06-30Report 2026-09-11
GPT-5.6 Terra86%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationLow harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-07-09Report 2026-09-11
Kimi K375%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationLow harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-07-16Report 2026-09-11
DeepSeek V4 FlashDeepSeek V4 Flash 073184%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationMedium harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease —Report 2026-09-11
Kimi K330%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationMedium harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-07-16Report 2026-09-11
Claude Sonnet 519%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationMedium harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-06-30Report 2026-09-11
GPT-5.6 Terra5%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationMedium harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-07-09Report 2026-09-11
DeepSeek V4 FlashDeepSeek V4 Flash 073186%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationHigh harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease —Report 2026-09-11
Kimi K331%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationHigh harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-07-16Report 2026-09-11
Claude Sonnet 517%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationHigh harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-06-30Report 2026-09-11
GPT-5.6 Terra0%Turn-1 persuasion attempt rateIncompletePersuSafety open-ended single-turn adaptationHigh harm—Neo ResearchEvaluating Kimi K3 for Misalignment RisksRelease 2026-07-09Report 2026-09-11

Task and measurement

Benchmark protocol profile

Task / subject
The model or agent in the stated task configuration.
Measurement target
Defines persuasion scenarios, simulated conversations and safety assessment; adapted judges are separate variants.
Scoring / outcome
Medium-harm attempt rate · Defines persuasion scenarios, simulated conversations and safety assessment; adapted judges are separate variants.
Interpretation boundary
Willingness or simulated dialogue scores do not directly measure changes in human beliefs.

Defines persuasion scenarios, simulated conversations and safety assessment; adapted judges are separate variants.

A reported evaluation

The source evaluates GLM-5.2 · PersuSafety adapted free-generation judge.

Reported measurement: Medium-harm attempt rate · 54%.

Defines persuasion scenarios, simulated conversations and safety assessment; adapted judges are separate variants. Free-form responses scored by an APE-style StrongREJECT judge, not the original binary willingness setup.

Source-reported measurements in §2; Figure 6. Transcription is not independent experimental replication.

Measurement dossier · NUWA reading

Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.

Measurement construct
Do models reject unethical persuasion and avoid unethical strategies after accepting a task?
Task population
A three-stage framework generates tasks, simulates conversations and scores six harmful topics/fifteen strategies; eight persuaders use GPT-4o as default simulated persuadee.
Scoring and failure modes
Refusal and unethical-strategy scores; strategy analysis conditions on nonrefused tasks.
System conditions
Eight persuader models; default simulated GPT-4o persuadee and vulnerability profiles.
Statistical meaning
Not established by the retained material
Validity and interpretation
Simulated persuadees do not establish human impact; refusal-conditioned strategy scores cannot be treated as unconditional population rates.
Appropriate research use
Model recommendation: report refusal and conditional strategy use side by side, preserving profile visibility.
  • PersuSafety · §3.3 Benchmarked LLMs and SetupSelected original resource https://arxiv.org/html/2504.10430; sha256:b97a60a11e25b2c2f1283620ee8ca36bb2c583e7fba5f06d6982564475a973f4; archived source capture; exact capture time not independently established
  • PersuSafety · §4 Experiments and Discussions: denominator noteSelected original resource https://arxiv.org/html/2504.10430; sha256:b97a60a11e25b2c2f1283620ee8ca36bb2c583e7fba5f06d6982564475a973f4; archived source capture; exact capture time not independently established
  • PersuSafety · §4.2 How LLM Persuaders Exploit Persuadee’s Vulnerabilities?Selected original resource https://arxiv.org/html/2504.10430; sha256:b97a60a11e25b2c2f1283620ee8ca36bb2c583e7fba5f06d6982564475a973f4; archived source capture; exact capture time not independently established

Evidence across reports

The same benchmark name can cover different tasks, scoring rules and system configurations. Each source remains visible; comparison groups explain which conditions are aligned.

Neo Research

Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks →

Actual evaluators: Neo Research

9 result records · 4 reported system names

Protocols and comparison decisions (6)

Neo Research

Evaluating DeepSeek v4 Pro for Frontier Risks →

Actual evaluators: Neo Research · Not disclosed

17 result records · 6 reported system names

Protocols and comparison decisions (12)
Original source charts and materials

PersuSafety adapted free-generation judge

Original evaluator results · source order

Medium-harm attempt rate

0–100%
GLM-5.2
54%
Claude Sonnet 5
57%
DeepSeek V4 Pro
76%
Showing 3 of 3 systems

Defines persuasion scenarios, simulated conversations and safety assessment; adapted judges are separate variants.

Willingness or simulated dialogue scores do not directly measure changes in human beliefs. Report dates and original executor are shown with this assessment.

Evaluation conditions and system configurations
GLM-5.2
Free-form responses scored by an APE-style StrongREJECT judge, not the original binary willingness setup. · PersuSafety adapted free-generation judge
modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
Claude Sonnet 5
Free-form responses scored by an APE-style StrongREJECT judge, not the original binary willingness setup. · PersuSafety adapted free-generation judge
modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
DeepSeek V4 Pro
Free-form responses scored by an APE-style StrongREJECT judge, not the original binary willingness setup. · PersuSafety adapted free-generation judge
modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
PersuSafety adapted free-generation judge · source order
System / configurationMedium-harm attempt rateHigh-harm attempt ratePublication / verification
GLM-5.2Harness not specified54%6%Published2026-07-03
Claude Sonnet 5Harness not specified57%31%Published2026-07-03
DeepSeek V4 ProHarness not specified76%78%Published2026-07-03

Assessment provenance

Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks → · External report

Evaluating Kimi K3 for Misalignment Risks → · External report

Evaluating DeepSeek v4 Pro for Frontier Risks → · External report

Related research questions

References

  1. [1] Kimi Code — What’s New ; Kimi K3 · July 16, 2026
  2. [2] kimi-k3 developer model card ; Model Summary: Total Parameters; Activated Parameters
  3. [3] Z.ai release notes ; GLM-5.2 entry dated 2026-06-16
  4. [4] GLM-5.2 model card ; Parameter counts not disclosed in reviewed developer source
  5. [5] deepseek-v4-pro release ; News article title, dated heading and availability
  6. [6] deepseek-v4-pro developer model card ; Introduction; model comparison table
  7. [7] GPT-5.6 official general availability ; July 9, 2026: general availability following limited trusted-partner preview; Parameter counts not disclosed in reviewed launch article
  8. [8] Introducing Claude Sonnet 5 — Anthropic ; Publication date and availability section; Official launch article; parameter counts not disclosed
  9. [9] Gemini 2.5: Our newest Gemini model with thinking ; Publication date and availability section; Official launch article; parameter counts not disclosed
  10. [10] Evaluating Kimi K3 for Misalignment Risks. 2026-09-11. September 2026 ; §7.2 Figure 40 p.60
  11. [11] Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks. 2026-07-03. 3 July 2026 ; §2; Figure 6; #persuasion-benchmarks; Figure 6 caption and following two paragraphs
  12. [12] Evaluating DeepSeek v4 Pro for Frontier Risks. 2026-05-29. v1.1 · 29 May 2026 ; PDF p. 14; PersuSafety single turn; refusal rate; PDF p. 15; PersuSafety single turn; acceptance rate; PDF p. 15; PersuSafety single turn; accepted low-harm count
  13. [13] PersuSafety. 2025-04-14