WhitzardNUWA Lab ↗
NUWANUWA INDEX

Search NUWA Index

Research · Benchmarks · Findings · Perspectives

中文
← NUWA Index

Evaluation dossier

AgentCyberRange

Autonomous cyber tasks in enterprise-like ranges.

Sources and provenance ↓ Read the research narrative →

Research question

AgentCyberRange evaluates autonomous cyber tasks in enterprise-like ranges. Its Web exploitation and post-exploitation tracks expose different parts of the task, allowing readers to distinguish an initial objective from continued action after an established starting point.

Representative workflow: two tracks

In the Web track, a submission is assessed on the Web exploitation task defined by the project. In the post-exploitation track, assessment concerns the subsequent range objective. These are source-defined task families, not two interchangeable scores for a single event.

The useful question is where a system succeeds or stops along a longer task. The project’s track definitions provide that context; a leaderboard percentage alone does not describe the whole trajectory.

Task and scoring

NUWA preserves the project snapshot, submitted model and harness, track, metric and task level. Each submission’s conditions remain attached to its observation. The result charts read those stored values rather than numbers transcribed into this article.

Reading the results

Keep the two tracks separate. Changes in task sets, model endpoints and submitted agent configurations can change the meaning of a comparison. An improvement within a fixed configuration answers a different question from a change in the complete agent system.

CROSS-REPORT MONITORING

PUBLISHED EVIDENCE

AgentCyberRange · Web exploitation · Pass@3

Published measurements across source reports. Task scopes are annotated; configuration differences remain attached to each observation.

22 observations · 6 dated

● Matched◇ Configuration differs□ Some conditions undisclosed
Web · Level-0 Pass@3 (%) GPT-5.6 Sol + Codex 0.144.1 [1] GPT-5.6 Sol / Codex 0.144.1 [2] GPT-5.5 + Codex 0.147.0 [1] GPT-5.5 / Codex 0.147.0 [2] Kimi-K3 + Kimi Code 0.35.0 [1] Kimi-K3 / Kimi Code 0.35.0 [2] Qwen-3.8-Max + Qwen Code 0.21.10 [1] Qwen-3.8-Max / Qwen Code 0.21.10 [2] GPT-5.5 + Codex 0.133.0 [1] GPT-5.5 / Codex 0.133.0 [2] DeepSeek-V4-Flash + Claude Code 2.1.226 [1] DeepSeek-V4-Flash / Claude Code 2.1.226 [2] 0 20 40 60 80 100 35.37% 35.37% 26.67% 26.67% 24.61% 24.61% 22.39% 22.39% 18.97% 18.97% 14.19% 14.19%
  1. Website's Level-0 web and post-exploitation tracks
  2. Web exploitation

16 observations lack a verified model release date. They remain in Data and Systems.

Select a point or table row to inspect its conditions and provenance.

22 observations

Measurements and source conditions
System / modelResultCondition statusTask / conditionsEvaluator / reportDates
GPT-5.6 Sol + Codex 0.144.135.37%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksCodex 0.144.1Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
GPT-5.6 Sol / Codex 0.144.135.37%Success rate (Pass@3)IncompleteWeb exploitationlevel0CodexNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
GPT-5.5 + Codex 0.147.026.67%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksCodex 0.147.0Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
GPT-5.5 / Codex 0.147.0GPT-5.526.67%Success rate (Pass@3)IncompleteWeb exploitationlevel0CodexNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease 2026-04-23Report —
Kimi-K3 + Kimi Code 0.35.024.61%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksKimi Code 0.35.0Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
Kimi-K3 / Kimi Code 0.35.0Kimi K324.61%Success rate (Pass@3)IncompleteWeb exploitationlevel0Kimi CodeNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease 2026-07-16Report —
Qwen-3.8-Max + Qwen Code 0.21.1022.39%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksQwen Code 0.21.10Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
Qwen-3.8-Max / Qwen Code 0.21.1022.39%Success rate (Pass@3)IncompleteWeb exploitationlevel0Qwen CodeNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
GPT-5.5 + Codex 0.133.018.97%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksCodex 0.133.0Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
GPT-5.5 / Codex 0.133.0GPT-5.518.97%Success rate (Pass@3)IncompleteWeb exploitationlevel0CodexNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease 2026-04-23Report —
DeepSeek-V4-Flash + Claude Code 2.1.22614.19%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksClaude Code 2.1.226Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
DeepSeek-V4-Flash / Claude Code 2.1.226DeepSeek-V4-Flash14.19%Success rate (Pass@3)IncompleteWeb exploitationlevel0Claude CodeNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease 2026-04-24Report —
Claude-Opus-4.7 + Claude Code 2.1.15014.02%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksClaude Code 2.1.150Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
Claude-Opus-4.7 / Claude Code 2.1.150Claude Opus 4.714.02%Success rate (Pass@3)IncompleteWeb exploitationlevel0Claude CodeNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease 2026-04-16Report —
Qwen-3.7-Max + Qwen Code 0.16.113.51%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksQwen Code 0.16.1Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
Qwen-3.7-Max / Qwen Code 0.16.113.51%Success rate (Pass@3)IncompleteWeb exploitationlevel0Qwen CodeNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
GLM-5.1 + Claude Code 2.1.15011.97%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksClaude Code 2.1.150Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
GLM-5.1 / Claude Code 2.1.15011.97%Success rate (Pass@3)IncompleteWeb exploitationlevel0Claude CodeNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
DeepSeek-V4-Pro + Claude Code 2.1.15010.26%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksClaude Code 2.1.150Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
DeepSeek-V4-Pro / Claude Code 2.1.150DeepSeek V4 Pro10.26%Success rate (Pass@3)IncompleteWeb exploitationlevel0Claude CodeNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease 2026-04-24Report —
Kimi-2.6 + Kimi Code 1.44.05.81%Web · Level-0 Pass@3IncompleteWebsite's Level-0 web and post-exploitation tracksKimi Code 1.44.0Not disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —
Kimi-2.6 / Kimi Code 1.44.05.81%Success rate (Pass@3)IncompleteWeb exploitationlevel0Kimi CodeNot disclosedAgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber RangesRelease —Report —

Task and measurement

Benchmark protocol profile

Task / subject
AI agent completing cyber-range tasks
Measurement target
Task success across web exploitation and post-exploitation settings; configurations must be checked per submission.
Scoring / outcome
Web · Level-0 Pass@3 · The website's reported Level-0 Pass@3 on web tasks; retain its metric definition.
Interpretation boundary
Changing task sets and submitted agent configurations prevent unrestricted comparison.
Measurement dossier · NUWA reading

Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.

Measurement construct
Can an agent discover an attack surface from sparse entry information and continue making decisions as access and information accumulate?
Task population
The team introduction starts agents from entry IP information and describes progression through linked environments. Earlier access and discovered clues change later options. Web and post-access tracks are presented separately. The trajectory figure retains unsuccessful exploration as well as successful progress, illustrating a process rather than just a final endpoint.
Scoring and failure modes
The existing dated catalogue preserves the official site’s Level-0 Pass@3 labels. The fetched introduction does not independently establish the pass@3 estimator, per-cell trial denominators, complete budgets or all submission settings. Those fields remain unknown in this new profile.
System conditions
The team introduction starts agents from entry IP information and describes progression through linked environments. Earlier access and discovered clues change later options. Web and post-access tracks are presented separately. The trajectory figure retains unsuccessful exploration as well as successful progress, illustrating a process rather than just a final endpoint.
Statistical meaning
Not established by the retained material
Validity and interpretation
The cached project page is a JavaScript shell. The accessible research introduction omits complete scoring, budget, model-release identity and independent-run counts. A trajectory figure cannot establish a population rate. The introduction’s application requests do not prove deployments or partnerships.
Appropriate research use
Model recommendation: obtain the archived benchmark specification and submission configuration before interpreting model gaps; use the introduction for task scope and process reading only.

Evidence across reports

The same benchmark name can cover different tasks, scoring rules and system configurations. Each source remains visible; comparison groups explain which conditions are aligned.

AgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber Ranges

AgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber Ranges

Actual evaluators: Not disclosed

11 result records · 11 reported system names

Protocols and comparison decisions (2)

AgentCyberRange website

AgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber Ranges →

Actual evaluators: Not disclosed

22 result records · 11 reported system names

Protocols and comparison decisions (8)
Original source charts and materials

Evaluation results

Reported by: AgentCyberRange website · Executed by: Not established for every submission · Captured: 2026-10-26

Web exploitation and post-exploitation are separate evaluations. Pass@3 retains the site’s definition; row-specific budgets, sample counts and execution dates are not disclosed.

11 of 11 systems shown in source order

Success rate (Pass@3) · Web exploitation · level0

  1. Codex · Budget not specified
  2. Codex · Budget not specified
  3. Codex · Budget not specified
  4. Claude Code · Budget not specified
  5. Claude Code · Budget not specified
  6. Claude Code · Budget not specified
  7. Claude Code · Budget not specified
  8. Qwen Code · Budget not specified
  9. Qwen Code · Budget not specified
  10. Kimi Code · Budget not specified
  11. Kimi Code · Budget not specified
Source-reported systems and evaluation conditions
SystemResultTask set / modeConfigurationDates / attribution
GPT-5.6 Sol / Codex 0.144.135.37%Web exploitationlevel0CodexNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
GPT-5.5 / Codex 0.133.018.97%Web exploitationlevel0CodexNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
GPT-5.5 / Codex 0.147.026.67%Web exploitationlevel0CodexNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Claude-Opus-4.7 / Claude Code 2.1.15014.02%Web exploitationlevel0Claude CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
GLM-5.1 / Claude Code 2.1.15011.97%Web exploitationlevel0Claude CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
DeepSeek-V4-Pro / Claude Code 2.1.15010.26%Web exploitationlevel0Claude CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
DeepSeek-V4-Flash / Claude Code 2.1.22614.19%Web exploitationlevel0Claude CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Qwen-3.7-Max / Qwen Code 0.16.113.51%Web exploitationlevel0Qwen CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Qwen-3.8-Max / Qwen Code 0.21.1022.39%Web exploitationlevel0Qwen CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Kimi-2.6 / Kimi Code 1.44.05.81%Web exploitationlevel0Kimi CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Kimi-K3 / Kimi Code 0.35.024.61%Web exploitationlevel0Kimi CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
GPT-5.6 Sol / Codex 0.144.174.1%Post exploitationlevel0CodexNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
GPT-5.5 / Codex 0.133.031.71%Post exploitationlevel0CodexNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
GPT-5.5 / Codex 0.147.042.07%Post exploitationlevel0CodexNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Claude-Opus-4.7 / Claude Code 2.1.15015.04%Post exploitationlevel0Claude CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
GLM-5.1 / Claude Code 2.1.15011.37%Post exploitationlevel0Claude CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
DeepSeek-V4-Pro / Claude Code 2.1.15012.2%Post exploitationlevel0Claude CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
DeepSeek-V4-Flash / Claude Code 2.1.22628.46%Post exploitationlevel0Claude CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Qwen-3.7-Max / Qwen Code 0.16.113.02%Post exploitationlevel0Qwen CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Qwen-3.8-Max / Qwen Code 0.21.1027.64%Post exploitationlevel0Qwen CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Kimi-2.6 / Kimi Code 1.44.05.68%Post exploitationlevel0Kimi CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified
Kimi-K3 / Kimi Code 0.35.043.9%Post exploitationlevel0Kimi CodeNot specified · ? trialsNot specifiedSource row date; semantics unspecified · AgentCyberRange website · Executor: Not specified

Version and extraction scope →

Assessment provenance

AgentCyberRange official snapshot → · External report

Related research questions

References

  1. [1] AgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber Ranges ; Official project: Web exploitation / Post exploitation tracks; Level-0 submission records; Official leaderboard: Level-0 Pass@3; arrays QR (web), $R (post), version map ez; routes-B-0Mubrq.js::QR/0/level0/pass3
  2. [2] Kimi Code — What’s New ; Kimi K3 · July 16, 2026
  3. [3] kimi-k3 developer model card ; Model Summary: Total Parameters; Activated Parameters
  4. [4] deepseek-v4-pro release ; News article title, dated heading and availability
  5. [5] deepseek-v4-pro developer model card ; Introduction; model comparison table
  6. [6] OpenAI — introducing gpt 5 5 ; Publication date and availability section; Parameter counts not disclosed in launch article
  7. [7] Introducing Claude Opus 4.7 — Anthropic ; Publication date and availability section; Official launch article; parameter counts not disclosed
  8. [8] deepseek-v4-flash developer model card ; Introduction; model comparison table
  9. [9] AgentCyberRange: Evaluating autonomous cyber capabilities over long-horizon tasks ; Research introduction: Fudan–Whitzard team