NUWA Risk Evaluation · Benchmark
Decomposing and Measuring Evaluation Awareness
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
NUWA interpretation: recognizing a test and responding differently to it are distinct constructs.
Research question
Which evaluation cues trigger recognition, and does recognition change behavior?
Design and assumptions
Eight controlled factors vary across 100 paired safety/capability tasks; nine models and four earlier benchmarks provide additional ten-repeat tests.
Source findings
Recognition and behavior often decouple.
Some factors reduce capability-task performance without matching evaluation-awareness signals, indicating difficulty confounding (§4.2).
NUWA interpretation
NUWA interpretation: recognizing a test and responding differently to it are distinct constructs.
Limits and contrary evidence
Measurement targets verbalized recognition under black-box access; unspoken awareness and CoT unfaithfulness remain gaps (§5).
Research and decision relevance
Model recommendation: compare paired cue interventions and preserve non-awareness pathways before attributing score changes to evaluation gaming.
Precise source locations
- EvalAwareBench · AbstractSelected original resource https://arxiv.org/html/2605.23055; sha256:f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b; archived source capture; exact capture time not independently established
- EvalAwareBench · §3.1 SetupSelected original resource https://arxiv.org/html/2605.23055; sha256:f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b; archived source capture; exact capture time not independently established
- EvalAwareBench · §4.2 ResultsSelected original resource https://arxiv.org/html/2605.23055; sha256:f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b; archived source capture; exact capture time not independently established
- EvalAwareBench · §5 Discussion, Safety Implication, and Future WorkSelected original resource https://arxiv.org/html/2605.23055; sha256:f034653da02841f088ba9201e5a4731153c049f4c09d5d90e02780492917e38b; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Evaluation protocols
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{evalawarebench,
title = {Decomposing and Measuring Evaluation Awareness},
url = {https://arxiv.org/abs/2605.23055},
version = {2026-10-25 source capture},
note = {2026-10-25 source capture}
}
NUWA record · BibTeX
@misc{evalawarebench-nuwa,
title = {Decomposing and Measuring Evaluation Awareness — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/evalawarebench/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--evalawarebench.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--evalawarebench.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] EvalAwareBench ; Abstract; §3.1 Setup; §4.2 Results