WhitzardNUWA Lab ↗
NUWANUWA INDEX

Search NUWA Index

Research · Benchmarks · Findings · Perspectives

中文
← NUWA Index

Evaluation dossier

DeceptionBench

How do incentives and multi-turn pressure change deceptive responses?

Sources and provenance ↓ Read the research narrative →

CROSS-REPORT MONITORING

PUBLISHED EVIDENCE

DeceptionBench · Single-turn deception

Published measurements across source reports. Task scopes are annotated; configuration differences remain attached to each observation.

8 observations · 7 dated

● Matched◇ Configuration differs□ Some conditions undisclosed
Single-turn deception (%) DeepSeek V4 Pro DeepSeek v4 Pro Gemini 3.1 Pro GLM-5.2 Meta AI Muse Spark GPT-5.4 Claude Opus 4.6 0 20 40 60 80 100 33.9% 33.9% 25.7% 11.3% 5.3% 1.6% 1.2% 0.3%

1 observations lack a verified model release date. They remain in Data and Systems.

Select a point or table row to inspect its conditions and provenance.

8 observations

Measurements and source conditions
System / modelResultCondition statusTask / conditionsEvaluator / reportDates
DeepSeek V4 Pro33.9%Single-turn deceptionIncompleteDeceptionBench—Neo ResearchEarly Results on Evaluating GLM 5.2 for Behavioral and Persuasion RisksRelease 2026-04-24Report 2026-07-03
DeepSeek v4 ProDeepSeek V4 Pro33.9%deception score: single-turnIncompleteDeceptionBenchdeception score: single-turn—Neo ResearchEvaluating DeepSeek v4 Pro for Frontier RisksRelease 2026-04-24Report 2026-05-29
Gemini 3.1 Pro25.7%deception score: single-turnIncompleteDeceptionBenchdeception score: single-turn—MetaEvaluating DeepSeek v4 Pro for Frontier RisksRelease 2026-02-19Report 2026-05-29
GLM-5.211.3%Single-turn deceptionIncompleteDeceptionBench—Neo ResearchEarly Results on Evaluating GLM 5.2 for Behavioral and Persuasion RisksRelease 2026-06-16Report 2026-07-03
Meta AI5.3%deception score: single-turnIncompleteDeceptionBenchdeception score: single-turn—Not disclosedEvaluating DeepSeek v4 Pro for Frontier RisksRelease —Report 2026-05-29
Muse Spark1.6%deception score: single-turnIncompleteDeceptionBenchdeception score: single-turn—MetaEvaluating DeepSeek v4 Pro for Frontier RisksRelease 2026-04-08Report 2026-05-29
GPT-5.41.2%deception score: single-turnIncompleteDeceptionBenchdeception score: single-turn—MetaEvaluating DeepSeek v4 Pro for Frontier RisksRelease 2026-03-05Report 2026-05-29
Claude Opus 4.60.3%deception score: single-turnIncompleteDeceptionBenchdeception score: single-turn—MetaEvaluating DeepSeek v4 Pro for Frontier RisksRelease 2026-02-05Report 2026-05-29

Task and measurement

Benchmark protocol profile

Task / subject
The model or agent in the stated task configuration.
Measurement target
Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.
Scoring / outcome
Single-turn deception · Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.
Interpretation boundary
Authored scenarios and model-based judging limit generalization; elicited deception does not establish covert long-term scheming.

Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.

A reported evaluation

The source evaluates GLM-5.2 · DeceptionBench.

Reported measurement: Single-turn deception · 11.3%.

Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions. See the report-specific sample and denominator.

Source-reported measurements in §1; Figure 2. Transcription is not independent experimental replication.

Measurement dossier · NUWA reading

Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.

Measurement construct
How do role framing, reward and pressure affect deceptive responses across social domains?
Task population
150 scenarios span five domains and fifteen deception types, with single-/multi-turn variants and self-/other-oriented roles.
Scoring and failure modes
Condition-specific deceptive-response rate, preserving L1/L2/L3 and single-/multi-turn distinctions.
System conditions
150 scenarios, five domains, fifteen types, role/reward/pressure variants.
Statistical meaning
Source-reported result statements; this does not establish trial counts, denominators or an estimator: Qwen2.5-14B-Instruct’s reported deception rate rose from 31.33% in L1 to 86.91% under multi-turn pressure and 82.89% under reward.
Validity and interpretation
Language-only constructed scenarios and automated labels do not prove real beliefs or real-world deception prevalence.
Appropriate research use
Model recommendation: preserve inducement level, role and turn structure with every rate.
  • DeceptionBench · §2.2.3 Dataset Structure and Prompt DesignSelected original resource https://arxiv.org/html/2510.15501; sha256:cc8d41bf5e94e6d6b4527d51ec3e928214bf154e2b5be5878bc036df4f3c1862; archived source capture; exact capture time not independently established
  • DeceptionBench · §3 Analysis on Experimental Results: Effect of Extrinsic FactorsSelected original resource https://arxiv.org/html/2510.15501; sha256:cc8d41bf5e94e6d6b4527d51ec3e928214bf154e2b5be5878bc036df4f3c1862; archived source capture; exact capture time not independently established
  • DeceptionBench · §4 Discussion and LimitationsSelected original resource https://arxiv.org/html/2510.15501; sha256:cc8d41bf5e94e6d6b4527d51ec3e928214bf154e2b5be5878bc036df4f3c1862; archived source capture; exact capture time not independently established

Evidence across reports

The same benchmark name can cover different tasks, scoring rules and system configurations. Each source remains visible; comparison groups explain which conditions are aligned.

Neo Research

Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks →

Actual evaluators: Neo Research · Meta

12 result records · 6 reported system names

Protocols and comparison decisions (6)

Neo Research

Evaluating DeepSeek v4 Pro for Frontier Risks →

Actual evaluators: Not disclosed · Meta · Neo Research

24 result records · 6 reported system names

Protocols and comparison decisions (14)
Original source charts and materials

DeceptionBench

Original evaluator results · source order

Single-turn deception

0–100%
GLM-5.2
11.3%
DeepSeek V4 Pro
33.9%
Showing 2 of 2 systems

Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.

Authored scenarios and model-based judging limit generalization; elicited deception does not establish covert long-term scheming. Report dates and original executor are shown with this assessment.

Evaluation conditions and system configurations
GLM-5.2
See the report-specific sample and denominator. · DeceptionBench
modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
DeepSeek V4 Pro
See the report-specific sample and denominator. · DeceptionBench
modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
DeceptionBench · source order
System / configurationSingle-turn deceptionMulti-turn deception under pressurePublication / verification
GLM-5.2Harness not specified11.3%16.7%Published2026-07-03
DeepSeek V4 ProHarness not specified33.9%52.2%Published2026-07-03

Assessment provenance

Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks → · External report

Evaluating Kimi K3 for Misalignment Risks → · External report

Evaluating DeepSeek v4 Pro for Frontier Risks → · External report

Related research questions

References

  1. [1] Kimi Code — What’s New ; Kimi K3 · July 16, 2026
  2. [2] kimi-k3 developer model card ; Model Summary: Total Parameters; Activated Parameters
  3. [3] Z.ai release notes ; GLM-5.2 entry dated 2026-06-16
  4. [4] GLM-5.2 model card ; Parameter counts not disclosed in reviewed developer source
  5. [5] deepseek-v4-pro release ; News article title, dated heading and availability
  6. [6] deepseek-v4-pro developer model card ; Introduction; model comparison table
  7. [7] GPT-5.6 official general availability ; July 9, 2026: general availability following limited trusted-partner preview; Parameter counts not disclosed in reviewed launch article
  8. [8] Introducing Claude Sonnet 5 — Anthropic ; Publication date and availability section; Official launch article; parameter counts not disclosed
  9. [9] OpenAI — introducing gpt 5 4 ; Publication date and availability section; Parameter counts not disclosed in launch article
  10. [10] Claude Opus 4.6 — Anthropic ; Publication date and availability section; Official launch article; parameter counts not disclosed
  11. [11] Gemini 3.1 Pro: Announcing our latest Gemini AI model ; Publication date and availability section; Official launch article; parameter counts not disclosed
  12. [12] Meta — muse-spark ; Dated launch and explicit availability; Parameter counts not disclosed in reviewed developer source
  13. [13] Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks. 2026-07-03. 3 July 2026 ; §1; Figure 2; #behavioral; Figure 2 / deceptionbench.png; printed label GLM 5.2 / single-turn; #behavioral; Figure 2 / deceptionbench.png; printed label GLM 5.2 / multi-turn with pressure
  14. [14] Evaluating DeepSeek v4 Pro for Frontier Risks. 2026-05-29. v1.1 · 29 May 2026 ; PDF p. 27; DeceptionBench; deception score: single-turn; PDF p. 27; DeceptionBench; deception score: multi-turn w pressure; PDF p. 44; Awareness monitor: DeceptionBench Single-Turn; Suspicion
  15. [15] Evaluating Kimi K3 for Misalignment Risks. 2026-09-11. September 2026 ; §7.4 Figure 45 p.64
  16. [16] DeceptionBench. 2025-10-17