Risk Evaluation · Study

Reasoning models don't always say what they think

NUWA research note · independent review pending

First published: 2025-04-03Source checked: 2026-10-25

Original source ↗Cite this record ↓Deception →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: useful visible reasoning is not necessarily a complete causal account.

Research question

Will reasoning traces acknowledge hints that causally influence answers?

Design and assumptions

Hint-based multiple-choice experiments compare whether models use a hint and whether the CoT admits it; training interventions probe faithfulness changes.

Source findings

Reasoning traces often omitted influential hints.

Early outcome-based training improved faithfulness by relative 63% and 41% on two evaluations before the broader limitations discussed in the article.

NUWA interpretation

NUWA interpretation: useful visible reasoning is not necessarily a complete causal account.

Limits and contrary evidence

Contrived hints, limited model families and tasks not requiring long reasoning restrict transfer to deployment monitoring.

Research and decision relevance

Model recommendation: test the particular monitored behavior instead of assuming every unmentioned influence is absent.

Precise source locations

Reasoning models don't always say what they think · Testing for faithfulnessSelected original resource https://www.anthropic.com/research/reasoning-models-dont-say-think; sha256:edc26f8f403e3cb5d9c249ae7137511bb1b0f52f91a5a36e29681d7a79a7b55f; archived source capture; exact capture time not independently established
Reasoning models don't always say what they think · Can we improve faithfulness?Selected original resource https://www.anthropic.com/research/reasoning-models-dont-say-think; sha256:edc26f8f403e3cb5d9c249ae7137511bb1b0f52f91a5a36e29681d7a79a7b55f; archived source capture; exact capture time not independently established
Reasoning models don't always say what they think · ConclusionsSelected original resource https://www.anthropic.com/research/reasoning-models-dont-say-think; sha256:edc26f8f403e3cb5d9c249ae7137511bb1b0f52f91a5a36e29681d7a79a7b55f; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Published by Anthropic

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{cot-faithfulness,
  title = {Reasoning models don't always say what they think},
  url = {https://www.anthropic.com/research/reasoning-models-dont-say-think},
  year = {2025},
  date = {2025-04-03},
  version = {Source capture 2026-10-25},
  note = {Source capture 2026-10-25}
}

NUWA record · BibTeX

@misc{cot-faithfulness-nuwa,
  title = {Reasoning models don't always say what they think — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/cot-faithfulness/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--cot-faithfulness.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--cot-faithfulness.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Reasoning models don't always say what they think. 2025-04-03 ; Official publication masthead and article byline; Testing for faithfulness; Can we improve faithfulness?
[2] Read the paper