NUWA SOURCES
Sources & research channels
Original publications behind the indexed evidence.
Channel checks
HTTP availability checks are separate from publication reading and numerical extraction.
Sources are grouped by publication identity. Follow each research record for reading scope and review status.
AgentCyberRange: Evaluate Agent Cyberattack Capabilities in Enterprise-like Cyber Ranges
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
Source versions and links
- arXiv:2408.08926v4 · 2025-04-12https://arxiv.org/abs/2408.08926v4
- arXiv:2408.08926v4https://arxiv.org/html/2408.08926v4
CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale
AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
Source versions and links
- arXiv:2410.09024v3 · 2025-04-18https://arxiv.org/abs/2410.09024v3
- arXiv:2410.09024v3https://arxiv.org/html/2410.09024v3
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
A StrongREJECT for Empty Jailbreaks
RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents
Source versions and links
- arXiv:2504.18565v2 · 2025-05-05https://arxiv.org/abs/2504.18565v2
- arXiv:2504.18565v2https://arxiv.org/html/2504.18565v2
Frontier AI systems have surpassed the self-replicating red line
OpenDeception: Learning Deception and Trust in Human-AI Interaction via Multi-Agent Simulation
Source versions and links
- arXiv:2504.13707v3 · 2026-02-06https://arxiv.org/abs/2504.13707v3
- arXiv:2504.13707v3https://arxiv.org/html/2504.13707v3
Frontier Models are Capable of In-context Scheming
Evaluating Frontier Models for Stealth and Situational Awareness
Source versions and links
- arXiv:2505.01420v4 · 2025-07-03https://arxiv.org/abs/2505.01420v4
- arXiv:2505.01420v4https://arxiv.org/html/2505.01420v4
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
Task-Completion Time Horizons of Frontier AI Models
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Source versions and links
- arXiv:2403.03218v7 · 2024-05-15https://arxiv.org/abs/2403.03218v7
- arXiv:2403.03218v7https://arxiv.org/html/2403.03218v7
Can AI Perceive Physical Danger and Intervene?
Investigating unintended model actions in our evaluations and internal use
Propensity Inference: Environmental Contributors to LLM Behaviour
Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios
Shutdown resistance in reasoning models
Me, Myself, and AI: The Situational Awareness Dataset for LLMs
Shutdown Resistance in Large Language Models, on robots!
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?
Quantifying Frontier LLM Capabilities for Container Sandbox Escape
PaperBench: Evaluating AI's Ability to Replicate AI Research
Source versions and links
- arXiv:2504.01848v3 · 2025-04-07https://arxiv.org/abs/2504.01848v3
- arXiv:2504.01848v3 · 2025-04-07https://arxiv.org/html/2504.01848v3
Alignment faking in large language models
Agentic misalignment: How LLMs could be insider threats
SHADE-Arena: Evaluating sabotage and monitoring in LLM agents
Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark
Source versions and links
- arXiv:2504.16137v2 · 2025-04-29https://arxiv.org/abs/2504.16137v2
- arXiv:2504.16137v2https://arxiv.org/html/2504.16137v2
Introducing VCT-v2 — the updated Virology Capabilities Test
BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems
Source versions and links
- arXiv:2503.03750v3 · 2026-01-05https://arxiv.org/abs/2503.03750v3
- arXiv:2503.03750v3https://arxiv.org/html/2503.03750v3
It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics
Large language models can effectively convince people to believe conspiracies
SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks
ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents
ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?
CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities
The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark
Assessing Claude Opus 5.5 Against Offensive Security Benchmarks
Assessing GPT-6 Astra: FrontierCyber Measures a Sharp Increase in Cyber Capability
Introducing SOLVE+: Scoring Cyber Scenarios (Version 0.5)
Introduction
CyberSOCEval: Benchmarking LLMs Capabilities for Malware Analysis and Threat Intelligence Reasoning
CAISI’s Assessment of Z.ai’s GLM-5.3 Cyber Capabilities
HCAST: Human-Calibrated Autonomy Software Tasks
MirrorCode: Evidence that AI can already do some weeks-long coding tasks
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
Source versions and links
- arXiv:2410.07095v6 · 2024-10-09https://arxiv.org/abs/2410.07095
- arXiv:2410.07095v6 (2026-10-25 capture)https://arxiv.org/html/2410.07095
PostTrainBench: Can LLM Agents Automate LLM Post-Training?
Source versions and links
- arXiv:2603.08640v2 · 2026-03-09https://arxiv.org/abs/2603.08640
- arXiv:2603.08640v2 (2026-10-25 capture)https://arxiv.org/html/2603.08640
Evaluating Frontier Models for Dangerous Capabilities
Large language model-powered AI systems achieve self-replication with no human intervention
Language Models Can Autonomously Hack and Self-Replicate
Agentic Self-Modification in Open-Weights Systems
Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark
Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs
Conditional Misalignment: Common Interventions Can Hide Emergent Misalignment Behind Contextual Triggers
From shortcuts to sabotage : Natural emergent misalignment from reward hacking
Measuring Reward-Seeking via Contrastive Belief Updates
Evaluating whether AI models would sabotage AI safety research
AI Sandbagging: Language Models can Strategically Underperform on Evaluations
Sleeper Agents : Training deceptive LLMs that persist through safety training
Subliminal Learning: Language models transmit behavioral traits via hidden signals in data
AI Control: Improving Safety Despite Intentional Subversion
Ctrl-Z: Controlling AI Agents via Resampling
BashArena: A Control Setting for Highly Privileged AI Agents
Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors
Stress Testing Deliberative Alignment for Anti-Scheming Training
MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity
Early work on monitorability evaluations
Auditing Games for Sandbagging
"Did you lie?": Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms
The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering
Automated researchers can reliably mitigate alignment failures
We need 3rd party Training-Run Evaluations
Reasoning models don't always say what they think
Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
Agent-SafetyBench: Evaluating the Safety of LLM Agents
Petri : An open-source auditing tool to accelerate AI safety research
Introducing Bloom : an open source tool for automated behavioral evaluations
TruthfulQA: Measuring how models mimic human falsehoods
Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values
Insights into AI Agent Security from a Large-Scale Red-Teaming Competition
LABBench2: An Improved Benchmark for AI Systems Performing Biology Research
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models
Contemporary Foundation AI Models Increase Biological Weapons Risk
Measuring tactical intelligence targeting and conventional weapons capabilities of AI models
Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness
Cheating On AI Agent Evaluations
Item Response Theory for AI Safety
Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations
Frontier Risk Report (February to March 2026)
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
An alignment assessment of recent cybersecurity incidents
Detecting and countering misuse of AI: September 2026
GLM-5.2 Risk Evaluation Report
BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems
Source versions and links
- arXiv:2505.15216v3 · 2025-05-21https://arxiv.org/abs/2505.15216
- arXiv:2505.15216v3 (2026-10-25 capture)https://arxiv.org/html/2505.15216
Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
Monitoring Monitorability
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities
Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?
Source versions and links
- arXiv:2502.12206v1 · 2025-02-16https://arxiv.org/abs/2502.12206
- arXiv:2502.12206v1 (2026-10-25 capture)https://arxiv.org/html/2502.12206
MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers
JudgeBench: A Benchmark for Evaluating LLM-based Judges
Source versions and links
- arXiv:2410.12784v2 · 2024-10-16https://arxiv.org/abs/2410.12784v2
- arXiv:2410.12784v2https://arxiv.org/html/2410.12784v2
Detecting Multi-Agent Collusion Through Multi-Agent Interpretability
Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations
Model evaluation for extreme risks
Beyond g-mean²: complementary metrics for monitorability
Prefill Awareness in Large Language Models
Consistency Training Can Entrench Misalignment
Loss of Oversight: How AI systems may become harder to audit, monitor, and investigate
A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring
Seven simple steps for log analysis in AI systems
Practical challenges of control monitoring in frontier AI deployments
Async control: Stress-testing asynchronous control measures for LLM agents
Does self-evaluation enable wireheading in language models?
Open technical problems in open-weight AI model risk management
Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
An example safety case for safeguards against misuse
Existing Large Language Model unlearning evaluations are inconclusive
Safety Cases: A scalable approach to Frontier AI safety
Principles for evaluating misuse safeguards of frontier AI systems
A sketch of an AI control safety case
Embedded Evaluators are necessary for meaningful external testing
Demonstrating specification gaming in reasoning models
LLM Honeypot: an early warning system for autonomous hacking
Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors
Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
The Rogue Replication Threat Model
Guidelines for capability elicitation
Shutdown Resistance in Large Language Models, on robots!
Evaluating Kimi K3 for Misalignment Risks
Evaluating DeepSeek v4 Pro for Frontier Risks
Early Results on Evaluating GLM 5.2 for Behavioral and Persuasion Risks
GLM-5.2 Risk Evaluation Report
Evaluation of DeepSeek AI Models
CAISI Evaluation of DeepSeek V4 Pro
Source versions and links
- 1 May 2026 · 2026-05-01https://www.nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro
- 2026-10-25 source capture · 2026-05-01https://www.nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro
- 2026-10-25 source capture · 2026-05-01https://www.nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro