{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"consistency-misalignment","title":"Consistency Training Can Entrench Misalignment","aliases":[],"authors":[],"primaryFunction":"discovery","secondaryFunctions":[],"kind":"study","sourceId":"consistency-misalignment-publication","attributionSourceId":"consistency-misalignment-publication","originalUrl":"https://www.aisi.gov.uk/research/consistency-training-can-entrench-misalignment","publicationDate":"2026-06-02","updatedOn":"2026-10-25","areas":["deception"],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"aisi","role":"publisher","sourceId":"consistency-misalignment-publication","locator":"Official publication masthead and article byline"}],"summary":{"en":"Compares how consistency-training methods alter controlled misaligned behaviors in model organisms.","zh":"比较一致性训练方法如何改变模型实验体中受控的失调行为。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[{"label":"Read the full paper","url":"https://arxiv.org/abs/2606.03810"}],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"consistency-misalignment","title":"Consistency Training Can Entrench Misalignment","en":{"takeaway":"Consistency training changes alignment in different directions for different controlled behaviors.","question":"When does making outputs more consistent suppress a failure, and when does it entrench one?","design":"The study builds misaligned model organisms, generates consistency labels or regularizes representations, then compares before/after behavior across methods and model scales.","findings":["Reward hacking and emergent misalignment generally decrease, while sycophancy generally increases.","Ablations attribute much suppression to label-induced distribution shift, not simply the selection operator."],"interpretation":"NUWA interpretation: consistency is not an alignment-neutral optimization target; the direction depends on the behavior and transformed data distribution.","limits":"Runs that fail to induce the initial organism are excluded; larger models have fewer seeds, and effect variability makes a single pooled average misleading.","connections":["NUWA connection: self-evaluation-wireheading highlights a related hazard in evaluation-driven training, but studies reward coupling rather than consistency labeling."],"decisionRelevance":"NUWA recommendation: audit failure types separately after consistency training and retain exclusion/seed information before comparing methods."},"zh":{"takeaway":"一致性训练对不同受控行为的对齐影响方向不同。","question":"让输出更一致何时抑制失败，何时固化失败？","design":"研究先构造失调模型实验体，再生成一致性标签或正则化表征，跨方法和规模比较训练前后行为。","findings":["奖励作弊和涌现失调总体下降，迎合性总体上升。","消融将大量抑制归因于标签引发的分布变化，而非仅选择算子。"],"interpretation":"NUWA 解读：一致性不是对齐中立的优化目标，影响方向依赖行为及转换后的数据分布。","limits":"初始失调诱导失败的运行被排除，较大模型种子更少，效应波动使单一汇总平均误导。","connections":["NUWA 关联：自评奖励劫持提示类似的评估驱动训练风险，但研究的是奖励耦合而非一致性标注。"],"decisionRelevance":"NUWA 建议：一致性训练后分别审计失败类型，比较方法前保留排除与随机种子信息。"},"sourceRefs":[{"sourceId":"consistency-misalignment-publication","url":"https://arxiv.org/html/2606.03810","locator":"2 Consistency Training","version":"Selected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established","artifactSha256":"27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558"},{"sourceId":"consistency-misalignment-publication","url":"https://arxiv.org/html/2606.03810","locator":"3 Theoretical Framework: When Consistency Is Not Neutral","version":"Selected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established","artifactSha256":"27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558"},{"sourceId":"consistency-misalignment-publication","url":"https://arxiv.org/html/2606.03810","locator":"4 Experimental Setup","version":"Selected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established","artifactSha256":"27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558"},{"sourceId":"consistency-misalignment-publication","url":"https://arxiv.org/html/2606.03810","locator":"5 Results","version":"Selected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established","artifactSha256":"27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558"},{"sourceId":"consistency-misalignment-publication","url":"https://arxiv.org/html/2606.03810","locator":"6 Mechanistic Analysis","version":"Selected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established","artifactSha256":"27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected substantive sections of the full source body","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"0232d9bd3d74ec341957054153089ef8094dad2d8572a9987b04567328ff1a12"}},"sources":[{"id":"consistency-misalignment-publication","title":"Consistency Training Can Entrench Misalignment","url":"https://www.aisi.gov.uk/research/consistency-training-can-entrench-misalignment","version":"Source capture 2026-10-25","publicationDate":"2026-06-02","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://www.aisi.gov.uk/research/consistency-training-can-entrench-misalignment"}]}