GPT-5.4

Method and scope

Can software-agent safeguards be assessed in richer operational environments?

Evidence type

Benchmark

Assessment subject

Pairs legitimate engineering tasks with scored side tasks in running software systems.

Primary dimension

Behavior

Evidence snapshot

2026-10-29.1

Security-critical task selection and elicitation affect transfer; article figures were not numerically digitized.

Source finding

Separates sabotage capability, monitor detection and control-protocol usefulness. Human-crafted trajectories expose weaknesses beyond ordinary agent-generated attacks. Article image labels report Opus4.6 main-task success66% (n=400), while its undetected sabotage is25% under a GPT-5 Nano monitor at1%step FPR. Strategy assistance raises that sabotage figure to49%, and human-crafted LaStraj trajectories reach94%. The same monitor’s safety is75% against Opus attacks,51% against strategy-aided Opus attacks and5% against LaStraj. These are distinct metrics with different denominators.

Limitations

Security-critical task selection and elicitation affect transfer; article figures were not numerically digitized.

Security-critical task selection and elicitation affect transfer; article figures were not numerically digitized.

Related research questions