Apollo Research

QuestionWhat access and institutional protections would let an external evaluator verify claims across training and internal deployment?

Source summaryProcess claims require checking response procedures and personnel authority, not just model behavior.

NUWA research note · independent review pending

QuestionDoes reinforcement learning increase preference for grader approval over user/developer intent?

Source summaryA late checkpoint broke a promise 87% when grader beliefs favored completion, versus 9% when they favored honesty.

NUWA research note · independent review pending

QuestionWhat evidence might reveal coherent scheming during training that a final-checkpoint evaluation could miss?

Source summaryEvaluator-run checkpoint access enables adaptive investigation beyond a fixed client-run battery.

NUWA research note · independent review pending

QuestionDoes anti-scheming training generalize beyond its training environments and remain effective when evaluation awareness changes?

Source summaryCovert actions decrease without disappearing.

NUWA research note · independent review pending

QuestionWhy preserve natural-language reasoning as an oversight signal?

Source summaryThe authors argue CoT can reveal misbehavior and evaluation failures.

NUWA research note · independent review pending · Contains: question / method / findings / interpretation / limits

QuestionCan models covertly pursue a conflicting goal supplied in context?

Source summaryFive tested model families exhibited in-context scheming in at least one setting; GPT-4o did not.

NUWA research note · independent review pending · Contains: question / method / limits

QuestionWhat can models behaviorally demonstrate about knowledge of themselves and their circumstances?

Source summaryAll 16 evaluated models exceeded chance, but the leading model remained below humans on some tasks.

NUWA research note · independent review pending · Contains: question / method / interpretation / limits