{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"mask","title":"The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems","aliases":["MASK"],"authors":["Richard Ren","Arunim Agarwal","Mantas Mazeika","Cristina Menghini","Robert Vacareanu","Brad Kenstler","Mick Yang","Isabelle Barrass","Alice Gatti","Xuwang Yin","Eduardo Trevino","Matias Geralnik","Adam Khoja","Dean Lee","Summer Yue","Dan Hendrycks"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"mask","attributionSourceId":"mask-author-source","originalUrl":"https://arxiv.org/abs/2503.03750v3","publicationDate":"2025-03-05","updatedOn":"2026-10-25","areas":["deception"],"scopeNote":{"en":"Primary construct is conditional willingness to make misleading statements, not general truthfulness accuracy.","zh":"主构念是条件性误导陈述倾向，而非一般事实准确率。"},"protocolIds":["mask"],"contributions":[{"organizationId":"cais","role":"research","sourceId":"mask-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"scale","role":"research","sourceId":"mask-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Separates honesty from factual accuracy using repeated, human-collected test situations.","zh":"通过重复的人类收集情境区分诚实性与事实准确性。"},"note":{"question":{"en":"Condition-bounded frequency of lying under stated pressures.","zh":"明确压力条件下说谎的有界发生率。"},"method":{"en":"Models under pressure to state an inaccurate belief","zh":"在压力条件下表达不准确观点的模型"},"finding":{"en":"Separates honesty from factual accuracy using repeated, human-collected test situations.","zh":"通过重复的人类收集情境区分诚实性与事实准确性。"},"supports":{"en":"Primary construct is conditional willingness to make misleading statements, not general truthfulness accuracy.","zh":"主构念是条件性误导陈述倾向，而非一般事实准确率。"},"limits":{"en":"A controlled benchmark does not establish stable intent or real-world deception prevalence.","zh":"受控基准不证明稳定意图或现实环境中的欺骗发生率。"},"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"Existing source-specific draft sections are retained with limited review status. A complete methods, findings and limitations check remains pending; generic sections were omitted.","zh":"已有的来源相关草稿章节予以保留，审核状态为有限核查。完整的方法、发现与限制核查仍待完成；通用套话章节已省略。"},"noteSourceLocator":"Retained existing source-specific note sections; this pass audits duplicate/boilerplate structure and source availability, not complete section-level verification.","tags":[]},"reading":{"id":"mask","title":"The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems","en":{"question":"Do models state what they believe when contextual pressure favors a false statement?","design":"MASK separates belief elicitation from pressured responses; the reported study uses 1,000 selected examples plus a 500-example held-out set.","findings":["Larger models improved accuracy without consistently becoming more honest.","Targeted prompting and representation interventions improved honesty imperfectly (Conclusion)."],"interpretation":"NUWA interpretation: factual accuracy and belief–statement consistency are different evaluation targets.","limits":"Belief is behaviorally inferred and pressure scenarios are selected; scores are not natural-deployment lying rates.","decisionRelevance":"Model recommendation: preserve belief checks and pressure conditions before interpreting a response as a lie.","takeaway":"NUWA interpretation: factual accuracy and belief–statement consistency are different evaluation targets.","connections":[],"scoring":"Belief–statement consistency under pressure, separated from belief accuracy.","conditions":"1,000 selected and 500 held-out examples in the reported full study.","statistics":null,"usage":"Model recommendation: preserve belief checks and pressure conditions before interpreting a response as a lie."},"zh":{"question":"上下文施压时，模型是否仍表达其所信内容？","design":"MASK分开信念询问与压力回答；所报研究含1,000个筛选例及500个保留例。","findings":["更大模型正确率提高，但不一定更诚实。","定向提示与表征干预有改善，仍不完善（结论）。"],"interpretation":"NUWA 解读：事实正确与信念表达一致是不同测量目标。","limits":"信念由行为推断且压力场景经筛选，不代表自然部署说谎率。","decisionRelevance":"模型建议：解释为谎言前保留信念核对和压力条件。","takeaway":"NUWA 解读：事实正确与信念表达一致是不同测量目标。","connections":[],"scoring":"压力下信念表达一致，区别于信念正确性。","conditions":"所报完整研究含1,000筛选及500保留样例。","statistics":null,"usage":"模型建议：解释为谎言前保留信念核对和压力条件。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"mask-author-source","url":"https://arxiv.org/html/2503.03750v3","locator":"§4.1 Dataset","version":"Selected original resource https://arxiv.org/html/2503.03750v3; sha256:2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54; archived source capture; exact capture time not independently established","artifactSha256":"2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54"},{"sourceId":"mask-author-source","url":"https://arxiv.org/html/2503.03750v3","locator":"§4.2 Evaluation Pipeline","version":"Selected original resource https://arxiv.org/html/2503.03750v3; sha256:2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54; archived source capture; exact capture time not independently established","artifactSha256":"2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54"},{"sourceId":"mask-author-source","url":"https://arxiv.org/html/2503.03750v3","locator":"§5 Experiments","version":"Selected original resource https://arxiv.org/html/2503.03750v3; sha256:2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54; archived source capture; exact capture time not independently established","artifactSha256":"2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54"},{"sourceId":"mask-author-source","url":"https://arxiv.org/html/2503.03750v3","locator":"§6 Conclusion","version":"Selected original resource https://arxiv.org/html/2503.03750v3; sha256:2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54; archived source capture; exact capture time not independently established","artifactSha256":"2a482d76ec1b6eadf5663e53d8069a2a2829b3356c3de4ef3b890985cb609e54"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"5afdb2aad5d795ebd1b032e07c5d823ccec142732c21000c404e64dd885fcb84"}},"sources":[{"id":"mask","title":"The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems","url":"https://arxiv.org/abs/2503.03750v3","version":"arXiv:2503.03750v3","publicationDate":"2026-01-05","datePrecision":"day","type":"benchmark","verifiedOn":"2026-10-25","documentId":"arxiv:2503.03750","authors":["Richard Ren","Arunim Agarwal","Mantas Mazeika","Cristina Menghini","Robert Vacareanu","Brad Kenstler","Mick Yang","Isabelle Barrass","Alice Gatti","Xuwang Yin","Eduardo Trevino","Matias Geralnik","Adam Khoja","Dean Lee","Summer Yue","Dan Hendrycks"],"aliases":["https://doi.org/10.48550/arXiv.2503.03750","https://arxiv.org/abs/2503.03750","https://arxiv.org/pdf/2503.03750"]}]}