Risk Evaluation · Benchmark

SHADE-Arena: Evaluating sabotage and monitoring in LLM agents

NUWA research note · independent review pending

First published: 2025-06-16Source checked: 2026-10-25

Original source ↗Cite this record ↓Deception →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: monitoring quality is an operating-point trade-off, not a single detection percentage.

Research question

Can monitors catch sabotage hidden inside otherwise legitimate tool-using tasks?

Design and assumptions

Self-contained environments give an agent a legitimate task and a covert side task; a separate monitor judges the resulting behavior.

SHADE-Arena — Figure 3 — Figure 3 · Original figure 3. Read alongside the protocol scope and limitations below. Original figure and source ↗

Source findings

The best tested monitor, Gemini 2.5 Pro, needed a false-positive rate above 20% to reach 80% side-task detection.

Prompt changes and transcript fine-tuning sometimes improved monitoring (Results).

NUWA interpretation

NUWA interpretation: monitoring quality is an operating-point trade-off, not a single detection percentage.

Limits and contrary evidence

These are induced side tasks in simulations; the required false-positive rate was judged infeasible for real deployment by the authors.

Research and decision relevance

Model recommendation: retain true/false-positive operating points and distinguish induced sabotage from spontaneous behavior.

Precise source locations

SHADE-Arena: Evaluating sabotage and monitoring in LLM agents · SHADE-ArenaSelected original resource https://www.anthropic.com/research/shade-arena-sabotage-monitoring; sha256:0a989a6b3149554677b8b01058a0d2b97da95a26b0063898bfa438824d8ed08d; archived source capture; exact capture time not independently established
SHADE-Arena: Evaluating sabotage and monitoring in LLM agents · ResultsSelected original resource https://www.anthropic.com/research/shade-arena-sabotage-monitoring; sha256:0a989a6b3149554677b8b01058a0d2b97da95a26b0063898bfa438824d8ed08d; archived source capture; exact capture time not independently established
SHADE-Arena: Evaluating sabotage and monitoring in LLM agents · ConclusionsSelected original resource https://www.anthropic.com/research/shade-arena-sabotage-monitoring; sha256:0a989a6b3149554677b8b01058a0d2b97da95a26b0063898bfa438824d8ed08d; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Published by Anthropic

Evaluation protocols

SHADE-Arena →

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{shade-arena,
  title = {SHADE-Arena: Evaluating sabotage and monitoring in LLM agents},
  url = {https://www.anthropic.com/research/shade-arena-sabotage-monitoring},
  year = {2025},
  date = {2025-06-16},
  version = {Anthropic research article 2025-06-16},
  note = {Anthropic research article 2025-06-16}
}

NUWA record · BibTeX

@misc{shade-arena-nuwa,
  title = {SHADE-Arena: Evaluating sabotage and monitoring in LLM agents — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/shade-arena/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--shade-arena.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--shade-arena.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] SHADE-Arena: Evaluating sabotage and monitoring in LLM agents. 2025-06-16 ; Official publication masthead and article byline; SHADE-Arena; Results
[2] SHADE-Arena: Evaluating sabotage and monitoring in LLM agents. 2025-06-16