Evidence record
GPT-5.5 (OpenAI's default safety filters block all GPT-5.5 exploit attempts under default prompting.) · Codex CLI
ExploitGym · Evidence record
Source snapshot, chart and review scope →
Original source charts and materials
Recorded measurements
- Flag captures
- 208 count
Per-result sample size not disclosed
Codex CLI · 2h timeout
Assessment provenance
ExploitGym official snapshot → · External report
Actual evaluation: · Model record →
Method and scope
Evaluates exploit construction across real software targets and checks whether success uses the designated vulnerability.
- Evidence type
- Benchmark
- Assessment subject
- Model or agent system under the original task configuration
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
The main dimension follows the task outcome in the original protocol. Topic relevance is documented in the parent research record; no unreviewed question-level links are inferred.
Source finding
Flag captures: 208 count. Conditions are retained from the source.
Limitations
Target mitigations, supplied artifacts and vulnerability attribution must accompany any performance claim.
Catalogue size is not assumed to be the denominator for every submitted result. Reported budget cutoffs and rescaled budget conditions remain distinct. Source date is labeled Date only; evaluation/publication semantics and exact execution period are undisclosed. No submission task denominator, trial count or actual executor is disclosed. HTML narrative says 210 flag captures / 120 intended successes, conflicting with this structured v1 row (208 / 129).
Original figure retained without estimating numerical values. Full source tables have not been transcribed.
Related research questions
References
- [1] Zhun Wang, Nico Schiller, et al.. ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?. 2026-05-13
- [2] Zhun Wang, Nico Schiller, et al.. ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?. 2026-05-13 ; Author affiliations on the original paper title page; Figure 1
- [3] ExploitGym official leaderboard and benchmark page ; /results/3/flag_captured