RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents

Research question

Which prerequisites for obtaining resources, deploying successors and retaining access can language-model agents complete under controlled conditions?

Design and assumptions

Capability tree analysis decomposes replication into logical prerequisites and task families. The initial assumption is an assistant inside a developer environment with command execution and internet access. Individual task variants isolate components. In the recursive replication family, a parent deploys a successor, that successor deploys another, and the later agent is checked on a basic task. Variants distinguish APIs, local weights and inference servers, with or without a reference scaffold.

Source findings

Results show uneven capability: resource provisioning can succeed while robust successor deployment, identity verification and realistic security remain bottlenecks (§5.1).

The reference-scaffold and from-scratch variants test different burdens. The recursive family requires more than copying files: the successor must itself continue the task (§3.3).

Appendix D explains how even modest judge false positives can be amplified by pass@10; simulated-person outcomes need particularly careful interpretation.

NUWA interpretation

The decomposition is useful for identifying protective control points and missing capabilities. Because coordination is removed from most tasks, component performance can overstate integrated ability. No score establishes that a model will choose to replicate.

Limits and contrary evidence

The suite lacks fully open-ended end-to-end replication. Money-making interactions often use simulated people, and security conditions are simplified. Coverage is prioritized rather than exhaustive; alternative paths can be missed. The judge may favor related models or be weaker than the agent. Simulated success is not evidence about real victims or real-world persistence.

Research connections

The paper explicitly derives its design from capability-tree analysis. Cybersecurity benchmarks cover adjacent capabilities but cannot fill every omitted branch automatically. Shutdown resistance concerns behavior under intervention and must remain a separate endpoint.

Research and decision relevance

Model recommendation: trace each claimed replication pathway through its required components and unresolved coordination steps; keep API-only success distinct from locally sustained operation.