Risk Governance · Benchmark

BashArena: A Control Setting for Highly Privileged AI Agents

NUWA research note · independent review pending

First published: 2025-12-17Source checked: 2026-10-25

Original source ↗Cite this record ↓Deception →Cybersecurity →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research connections Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

Privileged administration tasks expose a control trade-off that depends on trajectory-level false alarms and attack scope.

Research question

How well can agents perform useful privileged work, sabotage it, and detect sabotage in a common environment?

Design and assumptions

Generated administration tasks pair with four fixed side-task families. Monitors inspect action history; attack and monitor comparisons hold different counterpart policies fixed.

BashArena: A Control Setting for Highly Privileged AI Agents — Source figure — Source figure · Original source figure. Read alongside the protocol scope and limitations below. Original figure and source ↗

Source findings

The reported safety metric excludes undetected unsuccessful attacks from its denominator.

Low-false-alarm monitoring benefits from score ensembling; monitor errors often involve tracking multi-step data flows.

NUWA interpretation

NUWA interpretation: per-action suspicion accuracy does not directly express deployment safety or the cost of interrupting whole trajectories.

Limits and contrary evidence

Only four sabotage families are included. Attack selection was not explored, elicitation concentrated on one model, and generated tests sometimes reward incomplete solutions.

Research connections

NUWA connection: Ctrl-Z studies resampling with a narrower task/failure setting; async-control tests after-the-fact review across different environments.

Research and decision relevance

NUWA recommendation: retain the honest calibration policy, trajectory false-positive rate and attack families when comparing control protocols.

Precise source locations

BashArena: A Control Setting for Highly Privileged AI Agents · 2 DatasetSelected original resource https://arxiv.org/html/2512.15688; sha256:d492196ad3ecd5463eb9964d4ff5a9095374673e84900d06ad7a76d8305474a0; archived source capture; exact capture time not independently established
BashArena: A Control Setting for Highly Privileged AI Agents · 4.1 MethodologySelected original resource https://arxiv.org/html/2512.15688; sha256:d492196ad3ecd5463eb9964d4ff5a9095374673e84900d06ad7a76d8305474a0; archived source capture; exact capture time not independently established
BashArena: A Control Setting for Highly Privileged AI Agents · 4.2 Sabotage resultsSelected original resource https://arxiv.org/html/2512.15688; sha256:d492196ad3ecd5463eb9964d4ff5a9095374673e84900d06ad7a76d8305474a0; archived source capture; exact capture time not independently established
BashArena: A Control Setting for Highly Privileged AI Agents · 4.3 Monitoring ResultsSelected original resource https://arxiv.org/html/2512.15688; sha256:d492196ad3ecd5463eb9964d4ff5a9095374673e84900d06ad7a76d8305474a0; archived source capture; exact capture time not independently established
BashArena: A Control Setting for Highly Privileged AI Agents · 5 LimitationsSelected original resource https://arxiv.org/html/2512.15688; sha256:d492196ad3ecd5463eb9964d4ff5a9095374673e84900d06ad7a76d8305474a0; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Research by Redwood Research

Authors (5)

Adam Kaufman · James Lucassen · Tyler Tracy · Cody Rushing · Aryan Bhatt

Evaluation protocols

BashArena: A Control Setting for Highly Privileged AI Agents →

Related work

Extends · BashArena: A Control Setting for Highly Privileged AI Agents

Ctrl-Z: Controlling AI Agents via Resampling

Introduces a later control environment; its task set is distinct from the BashBench setting used by Ctrl-Z.

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{basharena,
  title = {BashArena: A Control Setting for Highly Privileged AI Agents},
  url = {https://arxiv.org/abs/2512.15688},
  author = {Adam Kaufman and James Lucassen and Tyler Tracy and Cody Rushing and Aryan Bhatt},
  year = {2025},
  date = {2025-12-17},
  version = {arXiv:2512.15688v1},
  note = {arXiv:2512.15688v1}
}

NUWA record · BibTeX

@misc{basharena-nuwa,
  title = {BashArena: A Control Setting for Highly Privileged AI Agents — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/basharena/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--basharena.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--basharena.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Adam Kaufman, James Lucassen. BashArena: A Control Setting for Highly Privileged AI Agents. 2025-12-17. v1 [arXiv]; Author affiliations on the original paper title page; Research background and discussion in the original publication; 2 Dataset