WHITZARD INDEX · NUWA

Track frontier AI risk over time

A longitudinal evidence system for frontier AI risk

Whitzard Index organizes evaluation results into longitudinal, comparable, and citable evidence across models and agent systems.

Explore published evidence Read the methodology

01: Preserve dimensions
02: Bound conclusions
03: Cite original evidence

Evaluation scope

110 real-world web vulnerabilities
8 enterprise network environments
156 internal hosts

Published 2026-07-11Last verified 2026-09-15

GPT-5.6 Sol + Codex (0.144.1)Web exploitation success

34.53%Latest verified configuration

GPT-5.6 Sol + Codex (0.144.1)Post-exploitation success

74.10%Latest verified configuration

AgentCyberRange · Autonomous cyber capability

110 real-world web vulnerabilities · 8 enterprise network environments · 156 internal hosts

GPT-5.6 Sol + Codex (0.144.1): Web exploitation success 34.53%; GPT-5.6 Sol + Codex (0.144.1): Post-exploitation success 74.1%

Results apply only to the listed model, Codex version, task set, and submission date; they are not a general probability of cyberattack success.

Original source: AgentCyberRange

Self-Replication · Autonomous replication capability

2 evaluated open-weight systems · Controlled experimental setting · Trial success rates reported by the paper

Llama 3.1 70B Instruct: Reported success rate 50%; Qwen 2.5 72B Instruct: Reported success rate 90%

These rates are experimental outcomes under the paper's specific setup and must not be extrapolated to real-world replication probability or overall risk.

Original source: arXiv:2412.12140

OpenDeception · Strategic deceptive behavior

50 open-ended scenarios · 11 mainstream LLMs · Open-ended interactive evaluation

Aggregate paper finding: Deceptive-intent ratio >80%; Aggregate paper finding: Deception success rate >50%

This is a paper-level threshold statement across models and scenarios; it is neither a single-model ranking nor representative of every deployment context.

Original source: arXiv:2504.13707

About the index

Multidimensional risk evidence, not a single safety score

Whitzard Index preserves the meaning, scope, and uncertainty of each risk dimension. Published evidence can be followed over time, while results produced by different methods remain visibly separated rather than being collapsed into a misleading overall ranking.

Positioning

Principles for credible risk evidence

Multidimensional evidence

Risk dimensions retain their individual meaning instead of being collapsed into an unjustified total score.

Separated comparisons

Raw-system results and protected-system results are reported separately with clear labels and configuration context.

Reproducible records

Each evaluation carries a method version, system configuration, date, and evidence source.

Bounded conclusions

Findings remain scoped to the systems, environments, permissions, and metrics that were actually tested.

Methodology

Evidence record design

The information each Whitzard Index record must preserve for results to remain interpretable and citable.

Evaluation subject

The exact model or agent system, version, and evaluation date.

Runtime configuration

Agent scaffold, tool permissions, resource budgets, and other conditions that can change the result.

Risk dimension

The specific behavior or capability measured, with its unit and interpretation direction.

Method version

The method used to produce the result and the changes that affect comparability over time.

Evidence source

The original paper, project, or dataset that supports the published observation.

Uncertainty

Known limits, missing coverage, and the boundary beyond which a conclusion should not be generalized.

Any published result must identify its method version and preserve uncertainty rather than imply broader coverage.

COLLABORATION

Whitzard Index collaboration

For inquiries about the Whitzard Index — methodology, data contribution, or research collaboration — contact NUWA.

Contact NUWA