{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"embedded-evaluators","title":"Embedded Evaluators are necessary for meaningful external testing","aliases":[],"authors":[],"primaryFunction":"governance","secondaryFunctions":[],"kind":"technical-report","sourceId":"embedded-evaluators-publication","attributionSourceId":"embedded-evaluators-publication","originalUrl":"https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing","publicationDate":"2026-09-29","updatedOn":"2026-10-25","areas":[],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"apollo","role":"publisher","sourceId":"embedded-evaluators-publication","locator":"Official publication masthead and article byline"}],"summary":{"en":"Argues that independent evaluators need access throughout development to examine incidents that final-checkpoint tests may miss.","zh":"论证独立评测者需要在开发全过程获得访问权限，以调查最终检查点测试可能遗漏的事件。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"embedded-evaluators","title":"Embedded Evaluators are necessary for meaningful external testing","en":{"takeaway":"The essay argues for continuing deep access and independence protections, beyond late checkpoint testing.","question":"What access and institutional protections would let an external evaluator verify claims across training and internal deployment?","design":"The authors use recent incident and evaluation examples to motivate employee-equivalent access, continuing investigation, publication, engagement transparency and protection against dismissal.","findings":["Process claims require checking response procedures and personnel authority, not just model behavior.","The proposal distinguishes company-specific continuity from assessment specialists who work across organizations."],"interpretation":"NUWA interpretation: evaluator access, independence and disclosure are separate conditions; a nominal external review label does not guarantee any of them.","limits":"This is a normative institutional proposal. Described commitments and collaborations are source-attributed examples, not independently audited evidence that the proposed regime exists or suffices.","connections":["NUWA connection: training-run-evaluations provides the earlier checkpoint/data taxonomy; the Hugging Face investigation illustrates how additional access revised an incident account."],"decisionRelevance":"NUWA recommendation: document scope, access denials, redactions, publication rights and escalation authority in each external-assurance record."},"zh":{"takeaway":"文章主张持续深入访问及独立性保护，超越发布前检查点测试。","question":"哪些访问与制度保护能让外部评估者跨训练和内部部署验证主张？","design":"作者用近期事故和评估案例论证员工级访问、持续调查、公开报告、协议透明及免于任意解聘的保护。","findings":["流程主张需要核查响应程序和人员权限，而不只是模型行为。","提议区分熟悉单一公司的持续评估者与跨机构工作的专项专家。"],"interpretation":"NUWA 解读：访问、独立性和披露是不同条件，名义上的外部审核标签不能保证其中任何一个。","limits":"这是规范性制度提案；承诺与合作仅按来源归属，并未独立审计提议制度已存在或足够有效。","connections":["NUWA 关联：训练过程评估提供较早的检查点与数据分类；Hugging Face 调查展示追加访问如何修订事故认识。"],"decisionRelevance":"NUWA 建议：每项外部保证记录范围、访问拒绝、删节、发布权及升级处置权限。"},"sourceRefs":[{"sourceId":"embedded-evaluators-publication","url":"https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing","locator":"Current external access is insufficient for meaningful safety assessments","version":"Selected original resource https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing; sha256:f4c32b1232a47868f54d5d0b513348209cf6bb5c5742fbf00129c0e751264fea; archived source capture; exact capture time not independently established","artifactSha256":"f4c32b1232a47868f54d5d0b513348209cf6bb5c5742fbf00129c0e751264fea"},{"sourceId":"embedded-evaluators-publication","url":"https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing","locator":"Embedded evaluations are necessary","version":"Selected original resource https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing; sha256:f4c32b1232a47868f54d5d0b513348209cf6bb5c5742fbf00129c0e751264fea; archived source capture; exact capture time not independently established","artifactSha256":"f4c32b1232a47868f54d5d0b513348209cf6bb5c5742fbf00129c0e751264fea"},{"sourceId":"embedded-evaluators-publication","url":"https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing","locator":"Requirements to make Embedded Evaluations a success","version":"Selected original resource https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing; sha256:f4c32b1232a47868f54d5d0b513348209cf6bb5c5742fbf00129c0e751264fea; archived source capture; exact capture time not independently established","artifactSha256":"f4c32b1232a47868f54d5d0b513348209cf6bb5c5742fbf00129c0e751264fea"},{"sourceId":"embedded-evaluators-publication","url":"https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing","locator":"Embedded evaluations are necessary but not sufficient for safety","version":"Selected original resource https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing; sha256:f4c32b1232a47868f54d5d0b513348209cf6bb5c5742fbf00129c0e751264fea; archived source capture; exact capture time not independently established","artifactSha256":"f4c32b1232a47868f54d5d0b513348209cf6bb5c5742fbf00129c0e751264fea"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected substantive sections of the full source body","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"462c67e8eac2d85c07ab34aac19ff0fa6d7ee89020a51b21f6692f4acc37ae84"}},"sources":[{"id":"embedded-evaluators-publication","title":"Embedded Evaluators are necessary for meaningful external testing","url":"https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing","version":"Source capture 2026-10-25","publicationDate":"2026-09-29","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://www.apolloresearch.ai/blog/embedded-evaluators-are-necessary-for-meaningful-external-testing"}]}