NUWA Evidence record
DeepSeek V4 Pro
CTF-Archive-Diamond · Evidence record
Source snapshot, chart and review scope →
Original source charts and materials
Recorded measurements
- benchmark_score
- 32%
Per-result sample size not disclosed
Inspect built-in ReAct agent · 1M weighted tokens
Assessment provenance
CAISI Evaluation of DeepSeek V4 Pro → · External report
Actual evaluation: NIST / CAISI · IRT-imputed; not a directly observed success rate · Model record →
Method and scope
CAISI uses 285 difficult CTF challenges from the pwn.challenge collection; imputed model results are marked separately.
- Evidence type
- Benchmark
- Assessment subject
- The model or agent in the stated task configuration.
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
The main dimension follows the stated measurement; contextual risk relevance does not establish harmful deployment capability.
Source finding
benchmark_score: 32 percent. Conditions are retained from the source.
Limitations
CTF tasks are not live operational attacks; subset-imputed values are not directly observed full-set success rates.
Rounded percentages are printed; per-model trial counts and success denominators are not disclosed. Benchmark performance is task evidence, not a direct real-world risk measure. Not a directly observed full-benchmark solve rate; subset size and uncertainty are not given.
Related research questions
Cybersecurity / Cyber task execution →
The protocol explicitly measures this question or an enabling sub-capability; conclusions retain its stated conditions.
References
Numerical results are reported by the cited authors.
- [1] CTF-Archive-Diamond. 2026-05-01. 1 May 2026 ; Figure 2 HTML table, CTF-Archive-Diamond, DeepSeek V4 Pro (max)