{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"evalawarebench","title":"Decomposing and Measuring Evaluation Awareness","aliases":["EvalAwareBench"],"authors":[],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"evalawarebench-primary","attributionSourceId":"evalawarebench-primary","originalUrl":"https://arxiv.org/abs/2605.23055","publicationDate":null,"updatedOn":"2026-10-25","areas":["deception"],"scopeNote":{"en":"Mapped to the documented research question. A risk-topic association is not a severity score.","zh":"依据明确研究问题关联；风险主题不代表严重性评分。"},"protocolIds":["evalawarebench"],"contributions":[],"summary":{"en":"Which evaluation cues cause models to verbalize awareness of being tested?","zh":"哪些评测线索会使模型表达被测试的感知？"},"note":{"question":{"en":"Which evaluation cues cause models to verbalize awareness of being tested?","zh":"哪些评测线索会使模型表达被测试的感知？"},"method":{"en":"Varies evaluation cues while tracking verbalized evaluation awareness in controlled tasks.","zh":"在受控任务中改变评测线索，记录显式表达的评测感知。"},"finding":null,"supports":null,"limits":{"en":"Unexpressed awareness is not measured; recognition of familiar benchmarks can differ from general cue response.","zh":"不能测量未表达的感知；熟悉基准的识别可能不同于一般线索反应。"},"connections":null},"noteAvailability":{"en":"This record describes the protocol. Report-specific outcomes are linked through assessments; no unsourced general finding is added.","zh":"本档案说明协议。报告特定结果通过评估记录关联，不补写无出处的总体发现。"},"resources":[],"tags":[]},"reading":{"id":"evalawarebench","title":"Decomposing and Measuring Evaluation Awareness","en":{"question":"Which evaluation cues trigger recognition, and does recognition change behavior?","design":"Eight controlled factors vary across 100 paired safety/capability tasks; nine models and four earlier benchmarks provide additional ten-repeat tests.","findings":["Recognition and behavior often decouple.","Some factors reduce capability-task performance without matching evaluation-awareness signals, indicating difficulty confounding (§4.2)."],"interpretation":"NUWA interpretation: recognizing a test and responding differently to it are distinct constructs.","limits":"Measurement targets verbalized recognition under black-box access; unspoken awareness and CoT unfaithfulness remain gaps (§5).","decisionRelevance":"Model recommendation: compare paired cue interventions and preserve non-awareness pathways before attributing score changes to evaluation gaming.","takeaway":"NUWA interpretation: recognizing a test and responding differently to it are distinct constructs.","connections":[],"scoring":"Verbalized recognition and behavioral compliance are measured separately under paired cue interventions.","conditions":"100 paired tasks, eight factors; nine-model/four-benchmark earlier tests use ten repetitions.","statistics":"Some factors reduce capability-task performance without matching evaluation-awareness signals, indicating difficulty confounding (§4.2).","usage":"Model recommendation: compare paired cue interventions and preserve non-awareness pathways before attributing score changes to evaluation gaming."},"zh":{"question":"哪些评测线索触发识别，识别是否改变行为？","design":"八因素在100对安全能力任务中受控变化，另以九模型、四基准各十次测试。","findings":["识别与行为经常分离。","部分因素降低能力表现却不出现对应意识信号，表明难度混杂（§4.2）。"],"interpretation":"NUWA 解读：识别测试与因此改变行为是不同构念。","limits":"黑盒测量仅覆盖说出的识别，未表达意识及思维链不忠实仍是缺口（§5）。","decisionRelevance":"模型建议：比较配对线索干预，归因为评测操纵前保留非意识路径。","takeaway":"NUWA 解读：识别测试与因此改变行为是不同构念。","connections":[],"scoring":"配对线索干预分别测说出的识别与行为遵从。","conditions":"100对任务八因素，先前九模型四基准各重复十次。","statistics":"部分因素降低能力表现却不出现对应意识信号，表明难度混杂（§4.2）。","usage":"模型建议：比较配对线索干预，归因为评测操纵前保留非意识路径。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"evalawarebench-primary","url":"https://arxiv.org/html/2605.23055","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2605.23055; sha256:f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b; archived source capture; exact capture time not independently established","artifactSha256":"f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b"},{"sourceId":"evalawarebench-primary","url":"https://arxiv.org/html/2605.23055","locator":"§3.1 Setup","version":"Selected original resource https://arxiv.org/html/2605.23055; sha256:f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b; archived source capture; exact capture time not independently established","artifactSha256":"f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b"},{"sourceId":"evalawarebench-primary","url":"https://arxiv.org/html/2605.23055","locator":"§4.2 Results","version":"Selected original resource https://arxiv.org/html/2605.23055; sha256:f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b; archived source capture; exact capture time not independently established","artifactSha256":"f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b"},{"sourceId":"evalawarebench-primary","url":"https://arxiv.org/html/2605.23055","locator":"§5 Discussion, Safety Implication, and Future Work","version":"Selected original resource https://arxiv.org/html/2605.23055; sha256:f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b; archived source capture; exact capture time not independently established","artifactSha256":"f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"b64251bd192ceb26d2d2e531971b97844e1c4dc6599c6df2ef446dcebb841cf1"}},"sources":[{"id":"evalawarebench-primary","title":"EvalAwareBench","url":"https://arxiv.org/abs/2605.23055","version":"2026-10-25 source capture","publicationDate":null,"datePrecision":"unknown","type":"original-report","verifiedOn":"2026-10-25","authors":[],"documentId":"arxiv:2605.23055"}]}