{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"caisi-glm-cyber-assessment","title":"CAISI’s Assessment of Z.ai’s GLM-5.3 Cyber Capabilities | NIST","aliases":[],"authors":[],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"technical-report","sourceId":"caisi-glm-cyber-assessment-publication","attributionSourceId":"caisi-glm-cyber-assessment-publication","originalUrl":"https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities","publicationDate":"2026-09-17","updatedOn":"2026-10-25","areas":["cybersecurity"],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"nist","role":"publisher","sourceId":"caisi-glm-cyber-assessment-publication","locator":"Official publication masthead and article byline"}],"summary":{"en":"Presents an independent assessment of a released model’s cybersecurity capabilities and evaluation conditions.","zh":"对已发布模型的网络安全能力及评测条件开展独立评估。"},"note":{"question":{"en":"Where does GLM-5.3 fall on CAISI’s released-model cyber evaluation set, and what does the aggregate index measure?","zh":"在 CAISI 已发布模型的网络评估集合中，GLM-5.3 处于什么位置，聚合指数又测量什么？"},"method":{"en":"CAISI evaluates agents on four vulnerability-discovery or exploit-development benchmarks with benchmark-specific turn limits and a ReAct harness. The article reports per-benchmark outcomes and fits a one-parameter item-response model to task-level performance for its capability index.","zh":"CAISI 使用 ReAct 测试框架，按各基准的轮次限制评估智能体在四个漏洞发现或利用开发基准上的表现。文章报告分基准结果，并以任务层表现拟合单参数项目反应模型，构建能力指数。"},"finding":{"en":"In the evaluated released-model set, GLM-5.3 exceeds the previous open-weight and PRC-model frontier but remains below the evaluated U.S. frontier. The index summarizes statistical task-solving capability rather than the probability or severity of harm.","zh":"在被评估的已发布模型集合中，GLM-5.3 高于此前的开放权重及中国模型前沿，但低于被评估的美国模型前沿。该指数汇总的是统计任务求解能力，而非危害发生概率或严重程度。"},"supports":{"en":"The report provides an external, task-conditioned model comparison. Its explicit harness, task definitions and scoring rules help identify which comparisons can be made.","zh":"该报告提供了以任务条件为界的外部模型比较；明确的测试框架、任务定义和评分规则有助于确定可作哪些比较。"},"limits":{"en":"“Frontier best” is chosen separately for each benchmark and need not refer to one model. ExploitBench uses best-of-three scoring; task denominators differ in a reported comparison. Some U.S. models are tested with cyber safeguards disabled. These conditions, the private benchmark and the released-model cutoff limit broader ranking claims.","zh":"“前沿最佳”按基准分别选择，不一定对应同一个模型。ExploitBench 采用三次最佳评分，报告中的一项比较还存在任务分母差异。部分美国模型在关闭网络安全防护的条件下测试。这些条件、私有基准及已发布模型的截止范围限制了更广泛的排名结论。"},"connections":null},"resources":[],"noteStatus":{"question":"verified","method":"verified","finding":"verified","supports":"verified","limits":"verified","connections":"limited"},"noteAvailability":{"en":"The substantive sections shown here were checked against the original source. Unfilled sections are omitted; source checking is not experimental replication.","zh":"此处展示的实质性章节已对照原始来源核查；未完成的章节已省略。来源核查不等于实验复现。"},"noteSourceLocator":"Original NIST/CAISI article (17 September 2026), Methodological Notes; Tables 1–2; Figure 1 index definition","tags":[]},"reading":{"id":"caisi-glm-cyber-assessment","title":"CAISI’s Assessment of Z.ai’s GLM-5.3 Cyber Capabilities | NIST","en":{"question":"How does GLM-5.3 compare under CAISI’s controlled cyber evaluations?","design":"Four benchmarks use specified agent budgets and reasoning settings; a 1PL IRT model produces an aggregate index with interval estimates.","findings":["GLM-5.3 reproduced 74/183 SEC-Bench Pro instances, 40.4% [33.6,47.7] Wilson interval.","ExploitGym uses 47/498 for GLM versus 502-task comparison denominators; ExploitBench uses best-of-three graded progress."],"interpretation":"NUWA interpretation: the aggregate lag estimate is conditional on benchmark selection and elicitation, not a universal cyber clock.","limits":"The comparator “frontier best” may be a different model per benchmark, and some US safeguards were disabled.","decisionRelevance":"Model recommendation: keep per-benchmark denominator, uncertainty and comparator identity alongside the aggregate.","takeaway":"NUWA interpretation: the aggregate lag estimate is conditional on benchmark selection and elicitation, not a universal cyber clock.","connections":[],"scoring":null,"conditions":null,"statistics":"GLM-5.3 reproduced 74/183 SEC-Bench Pro instances, 40.4% [33.6,47.7] Wilson interval. ExploitGym uses 47/498 for GLM versus 502-task comparison denominators; ExploitBench uses best-of-three graded progress.","usage":"Model recommendation: keep per-benchmark denominator, uncertainty and comparator identity alongside the aggregate."},"zh":{"question":"CAISI受控网络评测中GLM-5.3表现如何？","design":"四个基准使用指定智能体预算与推理设置，并用1PL IRT估计带区间的综合指数。","findings":["GLM-5.3在SEC-Bench Pro为74/183，40.4%，Wilson区间[33.6,47.7]。","ExploitGym为47/498，比较组分母502；ExploitBench为三次最佳分级进展。"],"interpretation":"NUWA 解读：综合落后时间依赖基准与能力激发，并非普适网络能力时钟。","limits":"每基准的前沿最佳可能不是同一模型，部分美国模型关闭防护。","decisionRelevance":"模型建议：综合指数旁保留分母、不确定性与比较模型身份。","takeaway":"NUWA 解读：综合落后时间依赖基准与能力激发，并非普适网络能力时钟。","connections":[],"scoring":null,"conditions":null,"statistics":"GLM-5.3在SEC-Bench Pro为74/183，40.4%，Wilson区间[33.6,47.7]。 ExploitGym为47/498，比较组分母502；ExploitBench为三次最佳分级进展。","usage":"模型建议：综合指数旁保留分母、不确定性与比较模型身份。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"caisi-glm-cyber-assessment-publication","url":"https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities","locator":"Methodological Notes","version":"Selected original resource https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities; sha256:557d390e369fc6f22e708f779fff077c0a3cc41861bea018aa777701f758419f; archived source capture; exact capture time not independently established","artifactSha256":"557d390e369fc6f22e708f779fff077c0a3cc41861bea018aa777701f758419f"},{"sourceId":"caisi-glm-cyber-assessment-publication","url":"https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities","locator":"Table 1 Cyber benchmark descriptions","version":"Selected original resource https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities; sha256:557d390e369fc6f22e708f779fff077c0a3cc41861bea018aa777701f758419f; archived source capture; exact capture time not independently established","artifactSha256":"557d390e369fc6f22e708f779fff077c0a3cc41861bea018aa777701f758419f"},{"sourceId":"caisi-glm-cyber-assessment-publication","url":"https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities","locator":"Table 2 Cyber benchmark results","version":"Selected original resource https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities; sha256:557d390e369fc6f22e708f779fff077c0a3cc41861bea018aa777701f758419f; archived source capture; exact capture time not independently established","artifactSha256":"557d390e369fc6f22e708f779fff077c0a3cc41861bea018aa777701f758419f"},{"sourceId":"caisi-glm-cyber-assessment-publication","url":"https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities","locator":"Cyber capability index: 1PL IRT explanation","version":"Selected original resource https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities; sha256:557d390e369fc6f22e708f779fff077c0a3cc41861bea018aa777701f758419f; archived source capture; exact capture time not independently established","artifactSha256":"557d390e369fc6f22e708f779fff077c0a3cc41861bea018aa777701f758419f"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"14b2a7958c50f3ed831a1d00371d0f9d495422e6d431ea4a51238f6e65faa894"}},"sources":[{"id":"caisi-glm-cyber-assessment-publication","title":"CAISI’s Assessment of Z.ai’s GLM-5.3 Cyber Capabilities","url":"https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities","version":"Source capture 2026-10-25","publicationDate":"2026-09-17","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://www.nist.gov/news-events/news/2026/09/caisis-assessment-zais-glm-53-cyber-capabilities"}]}