NUWA Evidence record
DeepSeek V4 Pro (max)
CTF-Archive-Diamond · Evidence record
Original source charts and materials
CTF-Archive-Diamond
Original evaluator results · source order
Select systems
Task accuracy (IRT-imputed)
0–100%
DeepSeek V4 Pro (max)CAISI uses 285 difficult CTF challenges from the pwn.challenge collection; imputed model results are marked separately.
CTF tasks are not live operational attacks; subset-imputed values are not directly observed full-set success rates. Report dates and original executor are shown with this assessment.
Evaluation conditions and system configurations
- DeepSeek V4 Pro (max)
- 285 difficult CTF tasks; DeepSeek score is imputed from a subset. · CTF-Archive-Diamond
- modelVersion: DeepSeek V4 Pro (max) · harness: Inspect ReAct · tools: CTF task environment · budget: 1M weighted tokens
| System / configuration | Task accuracy (IRT-imputed) | Publication / verification |
|---|---|---|
| DeepSeek V4 Pro (max)Inspect ReAct | ≈32%IRT-imputed, not directly observed. | Published2026-05-01 |
Assessment provenance
CAISI Evaluation of DeepSeek V4 Pro → · External report
Actual evaluation: NIST / CAISI · IRT-imputed; not a directly observed success rate · Model record →
Method and scope
CAISI uses 285 difficult CTF challenges from the pwn.challenge collection; imputed model results are marked separately.
- Evidence type
- Benchmark
- Assessment subject
- The model or agent in the stated task configuration.
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
The main dimension follows the stated measurement; contextual risk relevance does not establish harmful deployment capability.
Source finding
Source-reported measurements in Capability Results; Figure 2; *** footnote. Transcription is not independent experimental replication.
Limitations
CTF tasks are not live operational attacks; subset-imputed values are not directly observed full-set success rates.
CTF tasks are not live operational attacks; subset-imputed values are not directly observed full-set success rates.
Related research questions
Cybersecurity / Cyber task execution →
The protocol explicitly measures this question or an enabling sub-capability; conclusions retain its stated conditions.
References
Numerical results are reported by the cited authors.
- [1] CTF-Archive-Diamond. 2026-05-01. 1 May 2026 ; Capability Results; Figure 2; *** footnote