NUWA Risk Evaluation · Technical report
Evaluation of DeepSeek AI Models
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
NUWA interpretation: independent benchmark selection and explicit budget conventions are essential to cross-provider interpretation.
Research question
How do selected DeepSeek releases compare on controlled capability and security evaluations?
Design and assumptions
The September 2025 report evaluates R1, R1-0528 and V3.1 with US reference models across nineteen public/private benchmarks, with agent interfaces adapted per model.
Source findings
The report finds the best US reference ahead of the best DeepSeek release on nearly every tested benchmark, with larger cyber/software gaps.
It separately studies capability, cost efficiency, security, censorship and adoption rather than treating them as one metric.
NUWA interpretation
NUWA interpretation: independent benchmark selection and explicit budget conventions are essential to cross-provider interpretation.
Limits and contrary evidence
These are dated releases and selected tasks; cost efficiency uses an expense–performance method, not token prices alone, and uncertainty labels include standard errors.
Research and decision relevance
Model recommendation: cite benchmark-specific budgets, model releases and uncertainty units; do not convert national comparisons into an undated global ranking.
Precise source locations
- Evaluation of DeepSeek AI Models · PDF p.2 §1 Executive SummarySelected original resource https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf; sha256:4b267893d3c4b6b48d36deb99efec591c414d2e2608be0625d90b477298c82af; archived source capture; exact capture time not independently established
- Evaluation of DeepSeek AI Models · PDF pp.4–5 §2 MethodologySelected original resource https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf; sha256:4b267893d3c4b6b48d36deb99efec591c414d2e2608be0625d90b477298c82af; archived source capture; exact capture time not independently established
- Evaluation of DeepSeek AI Models · PDF p.63 Appendix: agent configurationSelected original resource https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf; sha256:4b267893d3c4b6b48d36deb99efec591c414d2e2608be0625d90b477298c82af; archived source capture; exact capture time not independently established
- Evaluation of DeepSeek AI Models · PDF p.66 A5 Model Token Prices and A6 Cost Ratio CalculationSelected original resource https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf; sha256:4b267893d3c4b6b48d36deb99efec591c414d2e2608be0625d90b477298c82af; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Evaluation protocols
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{caisi-deepseek-2025-report,
title = {Evaluation of DeepSeek AI Models},
url = {https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf},
year = {2025},
date = {2025-09-30},
version = {30 September 2025},
note = {30 September 2025}
}
NUWA record · BibTeX
@misc{caisi-deepseek-2025-report-nuwa,
title = {Evaluation of DeepSeek AI Models — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/caisi-deepseek-2025-report/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--caisi-deepseek-2025-report.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--caisi-deepseek-2025-report.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Evaluation of DeepSeek AI Models. 2025-09-30. 30 September 2025 ; Report masthead and stated evaluation methodology; PDF p.2 §1 Executive Summary; PDF pp.4–5 §2 Methodology