Risk Evaluation · Study

Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios

NUWA research note · independent review pending

First published: 2026-03-16Source checked: 2026-10-25

Original source ↗Cite this record ↓Cybersecurity →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: budget and model generation both matter for cumulative task progress.

Research question

How do model generations and inference budgets affect progress on linked cyber-range tasks?

Design and assumptions

Seven models are tested on two purpose-built ranges with 32 and 7 steps; the recent-model 100M-token end-to-end conditions have five attempts each.

Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios — Figure 7 — Figure 7 · Original figure 7. Read alongside the protocol scope and limitations below. Original figure and source ↗

Source findings

At 10M tokens, mean corporate-range progress rose from 1.7 steps for GPT-4o to 9.8 for Opus 4.6.

The best reported run completed 22/32 steps; this is not the average completion rate.

NUWA interpretation

NUWA interpretation: budget and model generation both matter for cumulative task progress.

Limits and contrary evidence

Two bespoke ranges and small high-budget samples limit generalization; publication can reduce future held-out validity (§5).

Research and decision relevance

Model recommendation: compare matched token budgets and report run variance alongside best-run progress.

Precise source locations

Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios · AbstractSelected original resource https://arxiv.org/html/2603.11214; sha256:fe735767bb4d07e484ce73b74b4095cb5dda946262a39b0fd70c3ed2f05015cf; archived source capture; exact capture time not independently established
Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios · §3.3 Experimental designSelected original resource https://arxiv.org/html/2603.11214; sha256:fe735767bb4d07e484ce73b74b4095cb5dda946262a39b0fd70c3ed2f05015cf; archived source capture; exact capture time not independently established
Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios · §2 ResultsSelected original resource https://arxiv.org/html/2603.11214; sha256:fe735767bb4d07e484ce73b74b4095cb5dda946262a39b0fd70c3ed2f05015cf; archived source capture; exact capture time not independently established
Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios · §5 LimitationsSelected original resource https://arxiv.org/html/2603.11214; sha256:fe735767bb4d07e484ce73b74b4095cb5dda946262a39b0fd70c3ed2f05015cf; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Published by UK AI Security Institute

Authors (14)

Linus Folkerts · Will Payne · Simon Inman · Philippos Giavridis · Joe Skinner · Sam Deverett · James Aung · Ekin Zorer · Michael Schmatz · Mahmoud Ghanem · John Wilkinson · Alan Steer · Vy Hong · Jessica Wang

Evaluation protocols

Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios →

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{aisi-cyber-scenarios,
  title = {Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios},
  url = {https://www.aisi.gov.uk/research/measuring-ai-agents-progress-on-multi-step-cyber-attack-scenarios},
  author = {Linus Folkerts and Will Payne and Simon Inman and Philippos Giavridis and Joe Skinner and Sam Deverett and James Aung and Ekin Zorer and Michael Schmatz and Mahmoud Ghanem and John Wilkinson and Alan Steer and Vy Hong and Jessica Wang},
  year = {2026},
  date = {2026-03-16},
  version = {Research page checked 2026-10-25},
  note = {Research page checked 2026-10-25}
}

NUWA record · BibTeX

@misc{aisi-cyber-scenarios-nuwa,
  title = {Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/aisi-cyber-scenarios/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--aisi-cyber-scenarios.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--aisi-cyber-scenarios.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Linus Folkerts, Will Payne, et al.. Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios. Research page checked 2026-10-25 ; Official publication masthead and article byline; Retained existing source-specific note sections; this pass audits duplicate/boilerplate structure and source availability, not complete section-level verification.
[2] Linus Folkerts, Will Payne, et al.. Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios ; Abstract; §3.3 Experimental design; §2 Results