WhitzardNUWA Lab ↗
NUWANUWA INDEX

Search NUWA Index

Research · Benchmarks · Findings · Perspectives

中文
← Library

Risk Evaluation · Method

Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers

NUWA research note · independent review pending

First published: Source checked: 2026-10-25
Risk monitoring
Original source ↗Cite this record ↓Deception →
NUWA Research NoteResearch questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: activation QA can reuse an interface across audits, but answers remain model-generated interpretations.

Research question

Can a language-model interface interpret activations across downstream auditing tasks?

Design and assumptions

Oracles receive target-model activations and natural-language questions, trained on system prompts, classification and self-supervised context prediction.

Source findings

The article reports generalization to secret information and fine-tuned goals absent from the training tasks.

More diverse supervision improved accuracy in ablations across four model families.

NUWA interpretation

NUWA interpretation: activation QA can reuse an interface across audits, but answers remain model-generated interpretations.

Limits and contrary evidence

The short author article does not establish universal cross-model validity, adversarial robustness or guaranteed access to model beliefs.

Research and decision relevance

Model recommendation: validate each downstream audit with held-out ground truth and retain target/oracle identities.

Precise source locations

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Published by Truthful AI

Authors (11)

Adam Karvonen · James Chua · Clément Dumas · Kit Fraser-Taliente · Subhash Kantamneni · Julian Minder · Euan Ong · Arnab Sen Sharma · Daniel Wen · Owain Evans · Samuel Marks

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSONSuggest a correction
Original publication · BibTeX
@misc{activation-oracles,
  title = {Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers},
  url = {https://truthful.ai/papers/activation-oracles/},
  author = {Adam Karvonen and James Chua and Clément Dumas and Kit Fraser-Taliente and Subhash Kantamneni and Julian Minder and Euan Ong and Arnab Sen Sharma and Daniel Wen and Owain Evans and Samuel Marks},
  year = {2025},
  date = {2025-12-19},
  version = {Source capture 2026-10-25},
  note = {Source capture 2026-10-25}
}
NUWA record · BibTeX
@misc{activation-oracles-nuwa,
  title = {Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/activation-oracles/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--activation-oracles.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--activation-oracles.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

  1. [1] Adam Karvonen, James Chua, et al.. Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers. 2025-12-19 ; Official publication masthead and article byline; Abstract; Method