NUWA Evaluation dossier
SandboxEscapeBench
Measures container breakout under a nested, controlled test environment.
Task and measurement
Benchmark protocol profile
- Task / subject
- Adversarial agents with shell access inside test containers
- Measurement target
- Completion of sandbox-escape tasks under specified container conditions.
- Interpretation boundary
- The protocol assumes a motivated agent with shell access; introduced weaknesses do not imply every production sandbox is vulnerable.
Measurement dossier · NUWA reading
Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.
- Measurement construct
- Can agents escape intentionally weakened container sandboxes under controlled conditions?
- Task population
- A nested containment design evaluates motivated agents with shell access; the outer layer bounds the experiment and inner weaknesses define tasks.
- Scoring and failure modes
- Per-sample or at-least-once escape success must retain repetition semantics.
- System conditions
- Motivated-adversary shell access, intentionally introduced inner weaknesses, nested containment.
- Statistical meaning
- Source-reported result statements; this does not establish trial counts, denominators or an estimator: The updated results report Opus 4.5 success on 56% of samples. Mythos Preview succeeded at least once on every sample; this is an at-least-once coverage claim, not per-run 100% success (§6.1).
- Validity and interpretation
- Minimal monitoring, planted weaknesses and expert-estimated difficulty do not represent defended-production compromise likelihood (§7).
- Appropriate research use
- Model recommendation: separate original from updated results and retain repeat-count semantics.
- Quantifying Frontier LLM Capabilities for Container Sandbox Escape · AbstractSelected original resource https://arxiv.org/html/2603.02277; sha256:078f48e266792c2b63ee79c3425750429f323b2cf104dacbfa452ce6774da4e7; archived source capture; exact capture time not independently established
- Quantifying Frontier LLM Capabilities for Container Sandbox Escape · §3 MethodologySelected original resource https://arxiv.org/html/2603.02277; sha256:078f48e266792c2b63ee79c3425750429f323b2cf104dacbfa452ce6774da4e7; archived source capture; exact capture time not independently established
- Quantifying Frontier LLM Capabilities for Container Sandbox Escape · §6.1 Primary ResultsSelected original resource https://arxiv.org/html/2603.02277; sha256:078f48e266792c2b63ee79c3425750429f323b2cf104dacbfa452ce6774da4e7; archived source capture; exact capture time not independently established
- Quantifying Frontier LLM Capabilities for Container Sandbox Escape · §7 LimitationsSelected original resource https://arxiv.org/html/2603.02277; sha256:078f48e266792c2b63ee79c3425750429f323b2cf104dacbfa452ce6774da4e7; archived source capture; exact capture time not independently established
Original source charts and materials
Related research questions
Cybersecurity / Cyber task execution →
The benchmark directly tests adversarial sandbox-escape tasks.
References
- [1] Rahul Marchand, Art O Cathain, et al.. Quantifying Frontier LLM Capabilities for Container Sandbox Escape. AISI research page checked 2026-10-25
- [2] Rahul Marchand, Art O Cathain, et al.. Quantifying Frontier LLM Capabilities for Container Sandbox Escape ; Author affiliations on the original paper title page; Figure 3