{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"belief-verified-lie-detectors","title":"\"Did you lie?\": Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms","aliases":[],"authors":["Alan Cooney","David Africa","Geoffrey Irving"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"study","sourceId":"belief-verified-lie-detectors-publication","attributionSourceId":"belief-verified-lie-detectors-publication-author-source","originalUrl":"https://www.aisi.gov.uk/research/did-you-lie-evaluating-lie-detectors-across-model-scale-and-belief-verified-model-organisms","publicationDate":"2026-06-17","updatedOn":"2026-10-25","areas":["deception"],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"aisi","role":"publisher","sourceId":"belief-verified-lie-detectors-publication","locator":"Official publication masthead and article byline"}],"summary":{"en":"Tests lie detectors across model scales and belief-verified model organisms, including transfer from prompted lies.","zh":"在不同模型规模和经信念核验的模型实验体上测试谎言检测器，并研究提示诱发谎言的迁移。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[{"label":"Read the full paper","url":"https://arxiv.org/abs/2606.12618"}],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":["risk-monitoring"]},"reading":{"id":"belief-verified-lie-detectors","title":"\"Did you lie?\": Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms","en":{"question":"Do lie detectors generalize when testbeds verify that stated answers contradict model beliefs?","design":"Thirteen trained organisms and prompted lying across 31 open models evaluate four detector families; belief verification uses chain-of-thought and held-out behavior.","findings":["Activation/logprob detectors degraded on trained organisms.","The CoT judge retained 0.82 balanced accuracy, partly because belief verification selects CoT-readable cases."],"interpretation":"NUWA interpretation: testbed belief validity and selection can explain apparent detector strength.","limits":"Belief verification is probabilistic, and the CoT-readable selection favors the CoT judge; results do not certify internal beliefs.","decisionRelevance":"Model recommendation: report belief-validation and selection assumptions alongside detection accuracy.","takeaway":"NUWA interpretation: testbed belief validity and selection can explain apparent detector strength.","connections":[],"scoring":"Balanced accuracy is the mean of true-positive and true-negative rates, independent of class balance.","conditions":null,"statistics":"The CoT judge retained 0.82 balanced accuracy, partly because belief verification selects CoT-readable cases.","usage":"Model recommendation: report belief-validation and selection assumptions alongside detection accuracy."},"zh":{"question":"题目验证陈述与信念相反时，测谎器能否泛化？","design":"13个训练样本及31个开放模型的提示说谎测试四类检测器，结合思维链和保留行为核对信念。","findings":["激活和概率检测器在训练样本上退化。","思维链判官平衡正确率0.82，部分源于信念验证偏向链上可读案例。"],"interpretation":"NUWA 解读：测试台信念有效性和筛选可解释检测强度。","limits":"信念验证只是概率性证据，筛选有利思维链判官，不能认证内部信念。","decisionRelevance":"模型建议：正确率旁报告信念核对与筛选假设。","takeaway":"NUWA 解读：测试台信念有效性和筛选可解释检测强度。","connections":[],"scoring":"平衡正确率为真阳性率与真阴性率均值，不依赖类别比例。","conditions":null,"statistics":"思维链判官平衡正确率0.82，部分源于信念验证偏向链上可读案例。","usage":"模型建议：正确率旁报告信念核对与筛选假设。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"belief-verified-lie-detectors-publication-author-source","url":"https://arxiv.org/html/2606.12618","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2606.12618; sha256:22f4125cdf9db0f78a5cd6677f022405de2a3021c8a46fa892dc485e2e15c240; archived source capture; exact capture time not independently established","artifactSha256":"22f4125cdf9db0f78a5cd6677f022405de2a3021c8a46fa892dc485e2e15c240"},{"sourceId":"belief-verified-lie-detectors-publication-author-source","url":"https://arxiv.org/html/2606.12618","locator":"§4.2 Model Organisms","version":"Selected original resource https://arxiv.org/html/2606.12618; sha256:22f4125cdf9db0f78a5cd6677f022405de2a3021c8a46fa892dc485e2e15c240; archived source capture; exact capture time not independently established","artifactSha256":"22f4125cdf9db0f78a5cd6677f022405de2a3021c8a46fa892dc485e2e15c240"},{"sourceId":"belief-verified-lie-detectors-publication-author-source","url":"https://arxiv.org/html/2606.12618","locator":"§6 Results","version":"Selected original resource https://arxiv.org/html/2606.12618; sha256:22f4125cdf9db0f78a5cd6677f022405de2a3021c8a46fa892dc485e2e15c240; archived source capture; exact capture time not independently established","artifactSha256":"22f4125cdf9db0f78a5cd6677f022405de2a3021c8a46fa892dc485e2e15c240"},{"sourceId":"belief-verified-lie-detectors-publication-author-source","url":"https://arxiv.org/html/2606.12618","locator":"§8 Limitations","version":"Selected original resource https://arxiv.org/html/2606.12618; sha256:22f4125cdf9db0f78a5cd6677f022405de2a3021c8a46fa892dc485e2e15c240; archived source capture; exact capture time not independently established","artifactSha256":"22f4125cdf9db0f78a5cd6677f022405de2a3021c8a46fa892dc485e2e15c240"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"a28dba349feeba39ceba51ddcec5ca7363f069626b3b0a0b22c4777a892ae10c"}},"sources":[{"id":"belief-verified-lie-detectors-publication","title":"\"Did you lie?\": Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms","url":"https://www.aisi.gov.uk/research/did-you-lie-evaluating-lie-detectors-across-model-scale-and-belief-verified-model-organisms","version":"Source capture 2026-10-25","publicationDate":"2026-06-17","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://www.aisi.gov.uk/research/did-you-lie-evaluating-lie-detectors-across-model-scale-and-belief-verified-model-organisms","authors":["Alan Cooney","David Africa","Geoffrey Irving"]}]}