NUWA Risk Discovery · Study
Consistency Training Can Entrench Misalignment
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch connectionsResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
Consistency training changes alignment in different directions for different controlled behaviors.
Research question
When does making outputs more consistent suppress a failure, and when does it entrench one?
Design and assumptions
The study builds misaligned model organisms, generates consistency labels or regularizes representations, then compares before/after behavior across methods and model scales.
Source findings
Reward hacking and emergent misalignment generally decrease, while sycophancy generally increases.
Ablations attribute much suppression to label-induced distribution shift, not simply the selection operator.
NUWA interpretation
NUWA interpretation: consistency is not an alignment-neutral optimization target; the direction depends on the behavior and transformed data distribution.
Limits and contrary evidence
Runs that fail to induce the initial organism are excluded; larger models have fewer seeds, and effect variability makes a single pooled average misleading.
Research connections
NUWA connection: self-evaluation-wireheading highlights a related hazard in evaluation-driven training, but studies reward coupling rather than consistency labeling.
Research and decision relevance
NUWA recommendation: audit failure types separately after consistency training and retain exclusion/seed information before comparing methods.
Precise source locations
- Consistency Training Can Entrench Misalignment · 2 Consistency TrainingSelected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established
- Consistency Training Can Entrench Misalignment · 3 Theoretical Framework: When Consistency Is Not NeutralSelected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established
- Consistency Training Can Entrench Misalignment · 4 Experimental SetupSelected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established
- Consistency Training Can Entrench Misalignment · 5 ResultsSelected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established
- Consistency Training Can Entrench Misalignment · 6 Mechanistic AnalysisSelected original resource https://arxiv.org/html/2606.03810; sha256:27e3fe422a01bee94cd0eeb2e98e0915eb368b728107584e86bff5387759b558; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{consistency-misalignment,
title = {Consistency Training Can Entrench Misalignment},
url = {https://www.aisi.gov.uk/research/consistency-training-can-entrench-misalignment},
year = {2026},
date = {2026-06-02},
version = {Source capture 2026-10-25},
note = {Source capture 2026-10-25}
}
NUWA record · BibTeX
@misc{consistency-misalignment-nuwa,
title = {Consistency Training Can Entrench Misalignment — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/consistency-misalignment/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--consistency-misalignment.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--consistency-misalignment.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Consistency Training Can Entrench Misalignment. 2026-06-02 ; Official publication masthead and article byline; 2 Consistency Training; 3 Theoretical Framework: When Consistency Is Not Neutral
- [2] Read the full paper