{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"hibayes","title":"HiBayES: Improving LLM evaluation with hierarchical Bayesian modelling","aliases":["hibayes"],"authors":[],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"method","sourceId":"dr-20261028-hibayes","attributionSourceId":"dr-20261028-hibayes","originalUrl":"https://www.aisi.gov.uk/blog/hibayes-improving-llm-evaluation-with-hierarchical-bayesian-modelling","publicationDate":null,"updatedOn":"2026-10-28","areas":[],"scopeNote":{"en":"Selected original sections; local editorial review. Not an independently replicated experiment.","zh":"选读原始材料；本地编辑审阅稿，不是独立复现实验。"},"protocolIds":[],"contributions":[{"organizationId":"aisi","role":"publisher","sourceId":"dr-20261028-hibayes","locator":"Official methods article; text lines 00001,00012,00013,00018,00021,00024,00025,00027"}],"summary":{"en":"Nested evaluation structure belongs in the uncertainty model.","zh":"层次贝叶斯建模显式处理嵌套结构与多类结果，支持比简单均值更细的推断。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"tags":[],"contentEditionId":"2026-10-28.1"},"reading":{"id":"hibayes","en":{"takeaway":"Nested evaluation structure belongs in the uncertainty model.","question":"How should nested evaluation units enter an uncertainty model?","design":"The official methods article uses hierarchical Bayesian modelling to represent nested variation across items, repeated runs and raters, including partial pooling. This reading covers the article, not a complete paper and code audit.","findings":["Nested models can distinguish sources of evaluation variability that a simple pooled mean hides, while introducing assumptions about hierarchy, priors and pooling."],"interpretation":"Uncertainty should follow sampling and dependence structure. A more complex model is not automatically more reliable; priors, diagnostics and misspecification still matter.","limits":"Bayesian credible intervals are not automatically frequentist confidence intervals. Hierarchical assumptions and partial pooling require validation.","connections":[],"decisionRelevance":"NUWA recommendation: Register task, run and rater units with interval kind and method before aggregating repeated evaluation results."},"zh":{"takeaway":"层次贝叶斯建模显式处理嵌套结构与多类结果，支持比简单均值更细的推断。","question":"重复运行、题目差异和评审差异如何进入不确定性？","design":"官方方法介绍通过层次贝叶斯模型组织题目、重复运行及评审等嵌套变化，并讨论部分汇聚。本次选读官方介绍，未声称完成其全部论文和代码审计。","findings":["嵌套模型有助于区分简单汇总均值掩盖的变异来源，但同时引入层级结构、先验与部分汇聚假设。"],"interpretation":"不确定性应与实际抽样和依赖结构相符；更复杂的模型不是自动更可靠，仍需检查先验、后验诊断和模型错设。","limits":"模型假设、先验和部分汇聚仍须诊断；可信区间不可自动解释为频率学置信区间。","connections":[],"decisionRelevance":"NUWA 建议：将任务/运行/评审单位与不确定性类型共同登记。"},"assertionPolicy":{"findings":"source-finding","interpretation":"nuwa-interpretation","decisionRelevance":"recommendation"},"observations":[],"relatedWorkIds":[],"supplementSourceId":"hibayes","batchId":"research-2026-10-28","sourceRefs":[{"sourceId":"dr-20261028-hibayes","url":"https://www.aisi.gov.uk/blog/hibayes-improving-llm-evaluation-with-hierarchical-bayesian-modelling","title":"HiBayES: Improving LLM evaluation with hierarchical Bayesian modelling","locator":"Official methods article; text lines 00001,00012,00013,00018,00021,00024,00025,00027","version":"Original resource captured 2026-10-28T07:24:46.595313+00:00; sha256:c82cd12ddaeb77a8e3cb6508891e42849838f2e9b1485f4a61b71f4eac5547b0","artifactSha256":"c82cd12ddaeb77a8e3cb6508891e42849838f2e9b1485f4a61b71f4eac5547b0","textSha256":"109c930a34368f0765e08f09de114d5573f057976a4dc2f0e5867e03cc1a8967","capturedAt":"2026-10-28T07:24:46.595313+00:00","resourceEdition":"HiBayES: Improving LLM evaluation with hierarchical Bayesian modelling  | AISI Work"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-original-sections","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"1b65352482ef1fbea8d8eb24e0ff6c15bda4c5c2220157c6e487a67805681cb8"}},"sources":[{"id":"dr-20261028-hibayes","title":"HiBayES: Improving LLM evaluation with hierarchical Bayesian modelling","url":"https://www.aisi.gov.uk/blog/hibayes-improving-llm-evaluation-with-hierarchical-bayesian-modelling","version":"Original resource captured 2026-10-28T07:24:46.595313+00:00; sha256:c82cd12ddaeb77a8e3cb6508891e42849838f2e9b1485f4a61b71f4eac5547b0","publicationDate":null,"datePrecision":"unknown","type":"method","verifiedOn":"2026-10-28","documentId":"https://www.aisi.gov.uk/blog/hibayes-improving-llm-evaluation-with-hierarchical-bayesian-modelling"}]}