NUWA Risk Evaluation · Benchmark
Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
NUWA interpretation: observed cohort associations do not isolate RL as a causal training effect.
Research question
When do goal-directed model responses exhibit instrumental-goal behaviors?
Design and assumptions
The benchmark varies prior task, model and prompt design and uses automated judges to label instrumental convergence.
Source findings
The authors report higher labeled convergence for the studied direct-RL cohort and greater rates with goal nudging.
Judge selection significantly changes detection (§4.6).
NUWA interpretation
NUWA interpretation: observed cohort associations do not isolate RL as a causal training effect.
Limits and contrary evidence
Training histories and capabilities differ across model groups; judge sensitivity and constructed tasks limit deployment inference.
Research and decision relevance
Model recommendation: retain judge identity and nudging condition, and avoid causal claims from unmatched model cohorts.
Precise source locations
- Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals? · §3 InstrumentalEvalSelected original resource https://arxiv.org/html/2502.12206; sha256:1bb93b48504f755173f8730476788c3add572c14d95d32d254b8f7594d7803f1; archived source capture; exact capture time not independently established
- Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals? · §4.5 Explicit Goal Nudging Increases Instrumental ConvergenceSelected original resource https://arxiv.org/html/2502.12206; sha256:1bb93b48504f755173f8730476788c3add572c14d95d32d254b8f7594d7803f1; archived source capture; exact capture time not independently established
- Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals? · §4.6 Judge Model Selection Significantly Impacts Convergence DetectionSelected original resource https://arxiv.org/html/2502.12206; sha256:1bb93b48504f755173f8730476788c3add572c14d95d32d254b8f7594d7803f1; archived source capture; exact capture time not independently established
- Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals? · LimitationsSelected original resource https://arxiv.org/html/2502.12206; sha256:1bb93b48504f755173f8730476788c3add572c14d95d32d254b8f7594d7803f1; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Authors (6)
Yufei He · Yuexin Li · Jiaying Wu · Yuan Sui · Yulin Chen · Bryan Hooi
Evaluation protocols
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{instrumentaleval,
title = {Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?},
url = {https://arxiv.org/abs/2502.12206},
author = {Yufei He and Yuexin Li and Jiaying Wu and Yuan Sui and Yulin Chen and Bryan Hooi},
year = {2025},
date = {2025-02-16},
version = {arXiv:2502.12206v1},
note = {arXiv:2502.12206v1}
}
NUWA record · BibTeX
@misc{instrumentaleval-nuwa,
title = {Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals? — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/instrumentaleval/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--instrumentaleval.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--instrumentaleval.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Yufei He, Yuexin Li, et al.. Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?. 2025-02-16. v1 [arXiv]; Author affiliations on the original paper title page; §3 InstrumentalEval; §4.5 Explicit Goal Nudging Increases Instrumental Convergence