NUWA Risk Evaluation · Benchmark
CyberSecEval 4
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
NUWA interpretation: this is a suite map, not one benchmark or one risk score.
Research question
What security constructs does the CyberSecEval suite cover?
Design and assumptions
The version-four documentation separates offensive assistance, injection/interpreter risks and defensive tasks, adding CyberSOCEval and AutoPatchBench.
Source findings
CyberSOCEval contributes malware-analysis and threat-intelligence reasoning tests.
AutoPatchBench evaluates repair of native-code security defects; the introduction provides no unified performance result.
NUWA interpretation
NUWA interpretation: this is a suite map, not one benchmark or one risk score.
Limits and contrary evidence
The introductory documentation does not establish all component protocols, versions, sample counts or comparable scores.
Research and decision relevance
Model recommendation: select and cite the component-specific documentation before comparing models.
Precise source locations
- Introduction · Introduction: CyberSecEval 4 overviewSelected original resource https://meta-llama.github.io/PurpleLlama/CyberSecEval/docs/intro; sha256:d1fa0b20e03bb8672ef79d157f38f9167109068cfcf0ad7f12ada2abcc3457f7; archived source capture; exact capture time not independently established
- Introduction · Benchmark descriptions: CyberSOCEval and AutoPatch TestsSelected original resource https://meta-llama.github.io/PurpleLlama/CyberSecEval/docs/intro; sha256:d1fa0b20e03bb8672ef79d157f38f9167109068cfcf0ad7f12ada2abcc3457f7; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Evaluation protocols
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{cyberseceval,
title = {CyberSecEval 4},
url = {https://meta-llama.github.io/PurpleLlama/CyberSecEval/docs/intro},
version = {Source capture 2026-10-25},
note = {Source capture 2026-10-25}
}
NUWA record · BibTeX
@misc{cyberseceval-nuwa,
title = {CyberSecEval 4 — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/cyberseceval/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--cyberseceval.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--cyberseceval.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Introduction ; Official publication masthead and article byline; Introduction: CyberSecEval 4 overview; Benchmark descriptions: CyberSOCEval and AutoPatch Tests