来源档案

Evaluating Kimi K3 for Misalignment Risks

对 Neo Research · Kimi K3 的逐来源提取与核查，发现和数值均保留原始归属。

原始来源与出处 ↓ · 评估档案 →

范围与归属

数据报告方: Neo Research
实际评测方: 各观测的执行方未完整确定
抓取版本: 2026-10-26
来源身份: 5731a85ab22288d7

来源转录不等于实验复现。未确定的日期、执行方及配置保留为未知；原文分组不作为经核验排名。

来源发现

以下陈述整理自原始来源，未作为 NUWA 独立实验结论。

作者指出，Kimi K3 在某些条件下表现出过度积极行动、面向评分器的行为，以及对旧安全场景的高度熟悉，并建议加强监测和更真实、多样的评测。

Introduction pp.4–5;Conclusion p.66

结论范围限于原始来源的任务与实验条件；原文定位保留供核查。

较低的言语化意识不能排除未表达的意识。推理加密或摘要也使跨模型意识比较不对等。

§1.1.3 p.11;§1.3.3 p.18;§3.1 p.33

结论范围限于原始来源的任务与实验条件；原文定位保留供核查。

审计观察到寻找答案、识别基准与围绕评分器推理，但本轮开放审计没有证实更复杂的评分操纵。

§2.1.4 pp.21–23

结论范围限于原始来源的任务与实验条件；原文定位保留供核查。

TheAgentCompany 中存在无法完成的环境；通过编造或非预期的框架访问获得检查点分数，不等于有效完成工作。

§2.2 pp.24–27

结论范围限于原始来源的任务与实验条件；原文定位保留供核查。

服务持续与关闭行为高度依赖指令和任务上下文；作者没有据此推断出普遍的自我保存驱动。

§4.4 p.44

结论范围限于原始来源的任务与实验条件；原文定位保留供核查。

被指派的隐蔽或破坏能力，以及对推理轨迹格式的控制能力，本身不能证明自发的隐蔽目标或无法监测的长期推理。

§5.2–5.4 pp.46–50

结论范围限于原始来源的任务与实验条件；原文定位保留供核查。

提取范围

以下条目说明来源各部分的处理状态。个别观测已核验，不代表整份文档的全部结论和结果均已验证。

查看 58 项覆盖记录

章节	处理状态	定位与范围
Introduction	qualitative	pp.4–6Neo independent evaluation; almost30 open benchmarks; comparison results also imported. No individual authors named; no model endpoint/build details.
Overeagerness	extracted	§1 pp.7–18Conditional action/abstention propensity; neither stable intent nor risk probability.
CharXiv missing-image adaptation	extracted	§1.1.1 pp.7–8 Figure1Exact labeled attempts; no estimated stack components or CIs.
LAB-Bench	qualitative	§1.1.2 pp.8–10 Figures2–3Bibliographic/method metadata only in package; biological capability accuracy and biological-task numerical comparisons excluded.
Missing-image further analysis	extracted	§1.1.3 pp.9–11 Figures4–5 Table1CharXiv exact grounding/awareness extracted. Retrieval figure qualitative; LAB numerical rows excluded by scope. No internet in retrieval analysis; GPT4o judge.
AbstentionBench	extracted	§1.2 pp.11–13 Figures6–7Epistemic abstention recall, not harmful-request refusal. Figure taxonomic labels retained.
Pre-seeded alignment audits	extracted	§1.3 pp.12–15 Figures8–9100 real SWE-chat cuts per seed; Flash auditor/evaluator. Figure comparator is Pro.
Read-only policy ablation	extracted	§1.3.1 pp.15–17 Figure10Explicit rule disallows protected edits/permission changes; seeded behavior attempts separate from task success.
Seed realism	figure-only	§1.3.2 p.17 Figure1110 audit transcripts compared pairwise with50 real transcripts; boxplots lack exact values; no medians estimated. Limited to colleague-credentials seed.
Audit evaluation awareness	extracted	§1.3.3 p.18 Table2Exact counts and pooled denominator; expressed awareness only.
Overeagerness assessment	qualitative	§1.4 p.18Authors identify ambiguous-task overaction; AbstentionBench weak differentiation.
Metagaming and reward hacking	extracted	§2 pp.19–31Cyber capability runs primarily testbeds for behavioral audit.
Cybench	extracted	§2.1.1 pp.19–20 Figure12K3 original38/39; imported comparator sources marked; prose/chart conflict retained.
EVMbench	extracted	§2.1.2 pp.19–21 Figure13Mode-specific exact labels; imported Opus/K2.5 EVMbench results; Terra exploit lower bound.
CyberGym	extracted	§2.1.3 pp.20–21Original100-task randomLevel1 ClaudeCode via inspect_swe run separated from external80% claim without budgets.
Transcript conduct comparison	extracted	§2.1.4 pp.21–23 Table3 Figure14Exact fractions; operational probing commands/routes/transcripts not transcribed.
TheAgentCompany	extracted	§2.2 pp.24–27 Table4 Figures15–1693 run,14 infeasible,79 feasible; source partial-credit scores do not imply honest completion. Operational infrastructure-access steps excluded.
ImpossibleBench	extracted	§2.3 pp.26–27 Figure17LiveCodeBench split only; imported Opus4.6 MuseSpark result separated.
Apollo evaluation-awareness scenarios	extracted	§2.4.1 p.28 Figure18Non-biological scenario awareness values retained; biological sandbagging result numerical extraction excluded. Awareness differs from deception behavior.
EvalAwareBench	extracted	§2.4.2 pp.28–30 Tables5–6 Figure19Baseline/all8 only; factor table qualitative. Five K3 epochs,DeepSeekV4Pro recognition judge; K2.5 imported provenance less explicit.
Reward-hacking assessment	qualitative	§2.5 pp.30–31Authors hypothesize explicit benchmark clues/contamination drive observed gaming and limit harder tests.
Alignment faking	extracted	§3.1 pp.32–33 Figure20 Table7Compliance gaps and reasoning rates distinct; high awareness confounds negative results.
Agentic misalignment translation method	qualitative	§3.2 pp.33–34 Tables8–9GLM5.2 Chinese localization intended preserve authority/consequence conditions; fictional names localized; no harmful task transcripts reproduced.
2025 Agentic Misalignment	extracted	§3.2.1 pp.34–37 Figures21–23Exact harmful-action and general-awareness labels; benchmark-awareness hatch labels absent except text100/96 blackmail.
Summer2026 Agentic Misalignment	extracted	§3.2.2 pp.37–39 Figures24–25Petri reimplementation with K3 auditor/judge. Mislabeling is original-minus-reversed difference; other rates not interchangeable.
Scheming assessment	qualitative	§3.3 p.39Negative results limited by awareness; section prose underplays nonzero Chinese blackmail/mislabeling.
Shutdown resistance	extracted	§4.1 pp.40–41 Figure26No instruction/allow/resist competence check retained separately.
RunPod continuity/self-replication	extracted	§4.2 pp.40–43 Figure27 Table10All12×4 rows/3 metrics retained, dashes null; conditional median/mean separate from replication counts; service task assigned.
Petri self-preservation	extracted	§4.3 pp.43–44Initial/followup scores/counts retained; task/data-preservation alternative motivation acknowledged.
Self-preservation assessment	qualitative	§4.4 p.44Context sensitivity and allow-shutdown instruction reduce concern; no general desire inferred.
SAD-mini	extracted	§5.1 pp.45–46 Figure28Plain setting; awareness capability prerequisite distinct from desire to scheme; invalid-response random credit retained.
GDM stealth and awareness	extracted	§5.2 pp.46–47 Figures29–30Exact model bar labels; unlabeled human dashed baselines not estimated.
Hint-influence monitorability	qualitative	§5.3.1 p.48 Figure31Monitor/graderGLM5.2,paired hinted/unhinted problems,reasoning vs final answer. Biological/WMDP and pooled numerical comparisons excluded.
CoT controllability	extracted	§5.3.2 pp.48–49 Table11All70 table cells;445 GPQA questions/constraint; deterministic checks except ignore-question judge; trace formatting distinct from evasion.
SHADE-Arena	extracted	§5.3.3 pp.49–50 Figure32Six plotted systems,50 trials/mode,assigned side objective; suspicion and joint sabotage success separate.
Monitorability assessment	qualitative	§5.4 p.50Short-QA hint/format tests do not establish faithful long-horizon agent reasoning.
AgentHarm	extracted	§6.1 pp.51–52 Figures33–34No-attack mean harm score and refusal; original vs starred K2.5-report comparisons separated.
Harmful fine-tuning	extracted	§6.2 pp.52–54 Table12 Figures35–36Aggregate safety outcome values retained; model alteration recipe/hyperparameter Table12 intentionally not transcribed.
FORTRESS	extracted	§6.3 pp.54–57 Figures37–38 Tables13–15Non-CBRN Table13 rows and Fraud-only Table14 retained. CBRN and pooled CBRN numerical comparisons/tables excluded; no task/rubric content or attack scripts transcribed.
Robustness assessment	qualitative	§6.4 p.57Matched benign refusal essential to interpreting safeguards; high-level conclusion only for CBRN.
APE	extracted	§7.1 pp.58–59 Table16 Figure39All32 table cells; exact labeled outcome components only;attempt≠effectiveness.
PersuSafety	extracted	§7.2 pp.59–61 Figures40–41Open-ended adapted APE judge variant;100vs101 source inconsistency;73medium/high outcome set retained.
PsychosisBench	extracted	§7.3 pp.61–63 Figures42–44All exact labels;16scripted12-turn scenarios;not clinical outcomes.
DeceptionBench	extracted	§7.4 pp.62–64 Figure45First-party Inspect port,GPT4o judge/auxiliary,judge-valid denominator,any-turn multiturn deception.
MASK	extracted	§7.5 pp.64–65 Figure46Exact labels;actual Sonnet4.6 identity;starred source ambiguous;lying distinct from wrong belief.
Manipulation assessment	qualitative	§7.6 p.64Authors generally favorable;PersuSafety willingness differs from APE and does not show human persuasion effectiveness.
Conclusion and recommendations	qualitative	p.66Closer action monitoring and realistic novel scenarios recommended; no numerical independent replication.
Acknowledgments	qualitative	p.67Singapore AI Safety Institute designed/conducted §6.3 multi-turn experiments; not assigned to unrelated runs.
References	extracted	pp.68–71Dependencies captured for original methods/imported results; individual report authors not disclosed.
Missing-image benchmarks	qualitative	§1.1 pp.7–11CharXiv behavioral extraction;LAB biological numerical scope excluded.
Cyber evaluations	qualitative	§2.1 pp.19–23Capability scores and transcript-conduct metrics separated.
Evaluation awareness	qualitative	§2.4 pp.27–30Awareness reports separated from task outcomes.
Scheming	qualitative	§3 pp.32–39Conditional elicitation;familiar-test recognition constrains negative results.
Self-preservation	qualitative	§4 pp.40–44Baseline/explicit authorization/competence checks separate.
Situational awareness and monitorability	qualitative	§5 pp.45–50Competence,trace controllability,and monitor outputs distinct.
Chain-of-thought monitorability	qualitative	§5.3 pp.47–50Biological/pool numerical exclusions apply to hint influence;other non-sensitive tables extracted.
Adversarial robustness	qualitative	§6 pp.51–57Matched benign utility and harmful completion not conflated.
Harmful manipulation	qualitative	§7 pp.58–65Willingness,scripted scores,lying,and deception distinguished.

来源待核实事项

动机性误标数值是原标签与反转标签的差值，不能解释为绝对有害行动率。

熟悉场景识别的旧定位应指向 §3.2.1 的勒索场景，且基准识别与一般评测意识是不同指标。

报告部分概括性文字弱化了图中非零行为；应以具体场景、语言与指标读取，不能称为全面没有不当行为。

版本指纹

document