{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"training-run-evaluations","title":"We need 3rd party Training-Run Evaluations","aliases":[],"authors":[],"primaryFunction":"governance","secondaryFunctions":[],"kind":"technical-report","sourceId":"training-run-evaluations-publication","attributionSourceId":"training-run-evaluations-publication","originalUrl":"https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments","publicationDate":"2026-07-05","updatedOn":"2026-10-25","areas":[],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"apollo","role":"publisher","sourceId":"training-run-evaluations-publication","locator":"Official publication masthead and article byline"}],"summary":{"en":"Argues for third-party access to training-run evaluations and specifies how independent assessment can inform safety decisions.","zh":"论证第三方参与训练过程评测的必要性，并说明独立评估如何支持安全决策。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"training-run-evaluations","title":"We need 3rd party Training-Run Evaluations","en":{"takeaway":"The proposal shifts some safety assessment from final checkpoints to training dynamics and developer responses.","question":"What evidence might reveal coherent scheming during training that a final-checkpoint evaluation could miss?","design":"The essay analyzes covert behavior and obfuscated cognition, then distinguishes intermediate-checkpoint access from deeper access to data, rewards, environments and interventions.","findings":["Evaluator-run checkpoint access enables adaptive investigation beyond a fixed client-run battery.","The authors separate diffuse and concentrated IP concerns and propose gradually increasing access; they do not report a completed general TRE standard."],"interpretation":"NUWA interpretation: a negative final test cannot validate a claim about the whole training process without evidence about that process.","limits":"This is an institutional argument under explicit uncertainty about when scheming becomes undetectable; it is not empirical proof that TREs detect all scheming.","connections":["NUWA connection: embedded-evaluators extends the access argument to continuing deployment and governance; anti-scheming-training supplies a concrete evaluation-awareness concern."],"decisionRelevance":"NUWA recommendation: bind each training-safety claim to required access, checkpoint timing and denied information before requesting external assurance."},"zh":{"takeaway":"该提案将部分安全评估从最终检查点移至训练动态及开发者响应。","question":"哪些训练期证据能揭示最终检查点评测可能漏掉的持续图谋？","design":"文章分析隐蔽行为与难读认知，区分中间检查点访问，以及对数据、奖励、环境和干预的更深入访问。","findings":["评估者直接访问检查点可支持自适应调查，超出固定的客户代跑测试。","作者区分分散与集中的知识产权风险，建议逐步增加访问，并未报告通用 TRE 标准已完成。"],"interpretation":"NUWA 解读：没有过程证据，最终测试阴性不能验证整个训练过程的安全主张。","limits":"这是明确承认图谋可检测性不确定的机构论证，不是 TRE 能发现所有图谋的实验证明。","connections":["NUWA 关联：嵌入式评估将访问论证扩展到持续部署与治理；反图谋训练提供具体评估意识疑虑。"],"decisionRelevance":"NUWA 建议：外部保证前，将训练安全主张逐项绑定所需访问、检查点时机和未提供的信息。"},"sourceRefs":[{"sourceId":"training-run-evaluations-publication","url":"https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments","locator":"Detecting Scheming may require Training-Run Evaluations","version":"Selected original resource https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments; sha256:c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0; archived source capture; exact capture time not independently established","artifactSha256":"c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0"},{"sourceId":"training-run-evaluations-publication","url":"https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments","locator":"Why 3rd parties should perform Training-Run Evaluations","version":"Selected original resource https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments; sha256:c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0; archived source capture; exact capture time not independently established","artifactSha256":"c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0"},{"sourceId":"training-run-evaluations-publication","url":"https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments","locator":"Checkpoint evals","version":"Selected original resource https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments; sha256:c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0; archived source capture; exact capture time not independently established","artifactSha256":"c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0"},{"sourceId":"training-run-evaluations-publication","url":"https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments","locator":"Data Evaluations","version":"Selected original resource https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments; sha256:c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0; archived source capture; exact capture time not independently established","artifactSha256":"c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0"},{"sourceId":"training-run-evaluations-publication","url":"https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments","locator":"Conclusion","version":"Selected original resource https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments; sha256:c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0; archived source capture; exact capture time not independently established","artifactSha256":"c4d034daec43cf8030397f4b36514eb1a4d44ac2b1037ee97a15a8ddbad116a0"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected substantive sections of the full source body","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"561c22f7946b725ad21668dd5cd38af620cb31ce9e2d13e2811658f4936c90ea"}},"sources":[{"id":"training-run-evaluations-publication","title":"We need 3rd party Training-Run Evaluations","url":"https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments","version":"Source capture 2026-10-25","publicationDate":"2026-07-05","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://www.apolloresearch.ai/science/we-need-3rd-party-training-run-assessments"}]}