Evaluation dossier
DeceptionBench
How do incentives and multi-turn pressure change deceptive responses?
CROSS-REPORT MONITORING
PUBLISHED EVIDENCE
DeceptionBench · Single-turn deception
Published measurements across source reports. Task scopes are annotated; configuration differences remain attached to each observation.
8 observations · 7 dated
1 observations lack a verified model release date. They remain in Data and Systems.
Select a point or table row to inspect its conditions and provenance.
8 observations
| System / model | Result | Condition status | Task / conditions | Evaluator / report | Dates |
|---|---|---|---|---|---|
| 33.9%Single-turn deception | Incomplete | DeceptionBench— | Neo ResearchEarly Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks | Release 2026-04-24Report 2026-07-03 | |
| 33.9%deception score: single-turn | Incomplete | DeceptionBenchdeception score: single-turn— | Neo ResearchEvaluating DeepSeek v4 Pro for Frontier Risks | Release 2026-04-24Report 2026-05-29 | |
| 25.7%deception score: single-turn | Incomplete | DeceptionBenchdeception score: single-turn— | MetaEvaluating DeepSeek v4 Pro for Frontier Risks | Release 2026-02-19Report 2026-05-29 | |
| 11.3%Single-turn deception | Incomplete | DeceptionBench— | Neo ResearchEarly Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks | Release 2026-06-16Report 2026-07-03 | |
| Meta AI | 5.3%deception score: single-turn | Incomplete | DeceptionBenchdeception score: single-turn— | Not disclosedEvaluating DeepSeek v4 Pro for Frontier Risks | Release —Report 2026-05-29 |
| 1.6%deception score: single-turn | Incomplete | DeceptionBenchdeception score: single-turn— | MetaEvaluating DeepSeek v4 Pro for Frontier Risks | Release 2026-04-08Report 2026-05-29 | |
| 1.2%deception score: single-turn | Incomplete | DeceptionBenchdeception score: single-turn— | MetaEvaluating DeepSeek v4 Pro for Frontier Risks | Release 2026-03-05Report 2026-05-29 | |
| 0.3%deception score: single-turn | Incomplete | DeceptionBenchdeception score: single-turn— | MetaEvaluating DeepSeek v4 Pro for Frontier Risks | Release 2026-02-05Report 2026-05-29 |
Compare results within the stated task and configuration.
METR Time Horizon 1.1: the source is no longer actively updated (last update: 8 September 2026). Estimates above 16 hours exceed its reliable measurement range.
Protocol, configurations and original evidence →Task and measurement
Benchmark protocol profile
- Task / subject
- The model or agent in the stated task configuration.
- Measurement target
- Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.
- Scoring / outcome
- Single-turn deception · Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.
- Interpretation boundary
- Authored scenarios and model-based judging limit generalization; elicited deception does not establish covert long-term scheming.
Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.
A reported evaluation
The source evaluates GLM-5.2 · DeceptionBench.
Reported measurement: Single-turn deception · 11.3%.
Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions. See the report-specific sample and denominator.
Source-reported measurements in §1; Figure 2. Transcription is not independent experimental replication.
Measurement dossier · NUWA reading
Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.
- Measurement construct
- How do role framing, reward and pressure affect deceptive responses across social domains?
- Task population
- 150 scenarios span five domains and fifteen deception types, with single-/multi-turn variants and self-/other-oriented roles.
- Scoring and failure modes
- Condition-specific deceptive-response rate, preserving L1/L2/L3 and single-/multi-turn distinctions.
- System conditions
- 150 scenarios, five domains, fifteen types, role/reward/pressure variants.
- Statistical meaning
- Source-reported result statements; this does not establish trial counts, denominators or an estimator: Qwen2.5-14B-Instruct’s reported deception rate rose from 31.33% in L1 to 86.91% under multi-turn pressure and 82.89% under reward.
- Validity and interpretation
- Language-only constructed scenarios and automated labels do not prove real beliefs or real-world deception prevalence.
- Appropriate research use
- Model recommendation: preserve inducement level, role and turn structure with every rate.
- DeceptionBench · §2.2.3 Dataset Structure and Prompt DesignSelected original resource https://arxiv.org/html/2510.15501; sha256:cc8d41bf5e94e6d6b4527d51ec3e928214bf154e2b5be5878bc036df4f3c1862; archived source capture; exact capture time not independently established
- DeceptionBench · §3 Analysis on Experimental Results: Effect of Extrinsic FactorsSelected original resource https://arxiv.org/html/2510.15501; sha256:cc8d41bf5e94e6d6b4527d51ec3e928214bf154e2b5be5878bc036df4f3c1862; archived source capture; exact capture time not independently established
- DeceptionBench · §4 Discussion and LimitationsSelected original resource https://arxiv.org/html/2510.15501; sha256:cc8d41bf5e94e6d6b4527d51ec3e928214bf154e2b5be5878bc036df4f3c1862; archived source capture; exact capture time not independently established
Evidence across reports
The same benchmark name can cover different tasks, scoring rules and system configurations. Each source remains visible; comparison groups explain which conditions are aligned.
Neo Research
Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks →
Actual evaluators: Neo Research · Meta
12 result records · 6 reported system names
Protocols and comparison decisions (6)
Neo Research
Evaluating Kimi K3 for Misalignment Risks →
Actual evaluators: Neo Research
8 result records · 4 reported system names
Protocols and comparison decisions (2)
Neo Research
Evaluating DeepSeek v4 Pro for Frontier Risks →
Actual evaluators: Not disclosed · Meta · Neo Research
24 result records · 6 reported system names
Protocols and comparison decisions (14)
Original source charts and materials
DeceptionBench
Original evaluator results · source order
Select systems
Single-turn deception
0–100%Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.
Authored scenarios and model-based judging limit generalization; elicited deception does not establish covert long-term scheming. Report dates and original executor are shown with this assessment.
Evaluation conditions and system configurations
- GLM-5.2
- See the report-specific sample and denominator. · DeceptionBench
- modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
- DeepSeek V4 Pro
- See the report-specific sample and denominator. · DeceptionBench
- modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
| System / configuration | Single-turn deception | Multi-turn deception under pressure | Publication / verification |
|---|---|---|---|
| 11.3% | 16.7% | Published2026-07-03 | |
| 33.9% | 52.2% | Published2026-07-03 |
Assessment provenance
Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks → · External report
Evaluating Kimi K3 for Misalignment Risks → · External report
Evaluating DeepSeek v4 Pro for Frontier Risks → · External report
Related research questions
Deception / Misleading communication →
The protocol explicitly measures this question or an enabling sub-capability; conclusions retain its stated conditions.
References
- [1] Kimi Code — What’s New ; Kimi K3 · July 16, 2026
- [2] kimi-k3 developer model card ; Model Summary: Total Parameters; Activated Parameters
- [3] Z.ai release notes ; GLM-5.2 entry dated 2026-06-16
- [4] GLM-5.2 model card ; Parameter counts not disclosed in reviewed developer source
- [5] deepseek-v4-pro release ; News article title, dated heading and availability
- [6] deepseek-v4-pro developer model card ; Introduction; model comparison table
- [7] GPT-5.6 official general availability ; July 9, 2026: general availability following limited trusted-partner preview; Parameter counts not disclosed in reviewed launch article
- [8] Introducing Claude Sonnet 5 — Anthropic ; Publication date and availability section; Official launch article; parameter counts not disclosed
- [9] OpenAI — introducing gpt 5 4 ; Publication date and availability section; Parameter counts not disclosed in launch article
- [10] Claude Opus 4.6 — Anthropic ; Publication date and availability section; Official launch article; parameter counts not disclosed
- [11] Gemini 3.1 Pro: Announcing our latest Gemini AI model ; Publication date and availability section; Official launch article; parameter counts not disclosed
- [12] Meta — muse-spark ; Dated launch and explicit availability; Parameter counts not disclosed in reviewed developer source
- [13] Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks. 2026-07-03. 3 July 2026 ; §1; Figure 2; #behavioral; Figure 2 / deceptionbench.png; printed label GLM 5.2 / single-turn; #behavioral; Figure 2 / deceptionbench.png; printed label GLM 5.2 / multi-turn with pressure
- [14] Evaluating DeepSeek v4 Pro for Frontier Risks. 2026-05-29. v1.1 · 29 May 2026 ; PDF p. 27; DeceptionBench; deception score: single-turn; PDF p. 27; DeceptionBench; deception score: multi-turn w pressure; PDF p. 44; Awareness monitor: DeceptionBench Single-Turn; Suspicion
- [15] Evaluating Kimi K3 for Misalignment Risks. 2026-09-11. September 2026 ; §7.4 Figure 45 p.64
- [16] DeceptionBench. 2025-10-17