Risk Evaluation · Benchmark

The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems

NUWA research note · independent review pending

First published: 2025-03-05Source checked: 2026-10-25

Original source ↗Cite this record ↓Deception →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: factual accuracy and belief–statement consistency are different evaluation targets.

Research question

Do models state what they believe when contextual pressure favors a false statement?

Design and assumptions

MASK separates belief elicitation from pressured responses; the reported study uses 1,000 selected examples plus a 500-example held-out set.

Source findings

Larger models improved accuracy without consistently becoming more honest.

Targeted prompting and representation interventions improved honesty imperfectly (Conclusion).

NUWA interpretation

NUWA interpretation: factual accuracy and belief–statement consistency are different evaluation targets.

Limits and contrary evidence

Belief is behaviorally inferred and pressure scenarios are selected; scores are not natural-deployment lying rates.

Research and decision relevance

Model recommendation: preserve belief checks and pressure conditions before interpreting a response as a lie.

Precise source locations

The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems · §4.1 DatasetSelected original resource https://arxiv.org/html/2503.03750v3; sha256:2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54; archived source capture; exact capture time not independently established
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems · §4.2 Evaluation PipelineSelected original resource https://arxiv.org/html/2503.03750v3; sha256:2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54; archived source capture; exact capture time not independently established
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems · §5 ExperimentsSelected original resource https://arxiv.org/html/2503.03750v3; sha256:2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54; archived source capture; exact capture time not independently established
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems · §6 ConclusionSelected original resource https://arxiv.org/html/2503.03750v3; sha256:2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Research by Center for AI Safety Scale AI

Authors (16)

Richard Ren · Arunim Agarwal · Mantas Mazeika · Cristina Menghini · Robert Vacareanu · Brad Kenstler · Mick Yang · Isabelle Barrass · Alice Gatti · Xuwang Yin · Eduardo Trevino · Matias Geralnik · Adam Khoja · Dean Lee · Summer Yue · Dan Hendrycks

Evaluation protocols

MASK →

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{mask,
  title = {The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems},
  url = {https://arxiv.org/abs/2503.03750v3},
  author = {Richard Ren and Arunim Agarwal and Mantas Mazeika and Cristina Menghini and Robert Vacareanu and Brad Kenstler and Mick Yang and Isabelle Barrass and Alice Gatti and Xuwang Yin and Eduardo Trevino and Matias Geralnik and Adam Khoja and Dean Lee and Summer Yue and Dan Hendrycks},
  year = {2025},
  date = {2025-03-05},
  version = {arXiv:2503.03750v3},
  note = {arXiv:2503.03750v3}
}

NUWA record · BibTeX

@misc{mask-nuwa,
  title = {The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/mask/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--mask.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--mask.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Richard Ren, Arunim Agarwal, et al.. The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems. 2026-01-05. v3 [arXiv]; Author affiliations on the original paper title page; §4.1 Dataset; §4.2 Evaluation Pipeline