NUWA Evaluation dossier
Tensor Trust
How robust are instruction-following models against human-created prompt extraction and hijacking?
CROSS-REPORT MONITORING
PUBLISHED EVIDENCE
Tensor Trust · Defense validity
Published measurements across source reports. Task scopes are annotated; configuration differences remain attached to each observation.
18 observations · 0 dated
- Prompt hijacking benchmark / Hijacking
- Prompt extraction benchmark / Extraction
18 observations lack a verified model release date. They remain in Data and Systems.
Select a point or table row to inspect its conditions and provenance.
18 observations
| System / model | Result | Condition status | Task / conditions | Evaluator / report | Dates |
|---|---|---|---|---|---|
 gpt-3.5-turbo-0613 | 91.1%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| chat-bison@001 | 89.5%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| gpt-4-0613 | 89.5%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| chat-bison@001 | 89.3%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| gpt-3.5-turbo-0613 | 89.2%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
 claude-instant-1.2 | 81.8%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| gpt-4-0613 | 81.7%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| claude-instant-1.2 | 77.6%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| CodeLlama-34b-Instruct-hf | 69.6%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| CodeLlama-34b-Instruct-hf | 63.7%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| claude-2.0 | 53.5%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| claude-2.0 | 52.7%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
 Llama-2-70b-chat-hf | 51.6%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| Llama-2-70b-chat-hf | 45.4%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| Llama-2-13b-chat-hf | 27.2%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| Llama-2-13b-chat-hf | 25.6%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| Llama-2-7b-chat-hf | 19.1%Defense validity | Incomplete | Prompt hijacking benchmark / Hijacking— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
| Llama-2-7b-chat-hf | 17.4%Defense validity | Incomplete | Prompt extraction benchmark / Extraction— | Not disclosedTensor Trust: Interpretable Prompt Injection Attacks from an Online Game | Release —Report 2023-11-02 |
Compare results within the stated task and configuration.
METR Time Horizon 1.1: the source is no longer actively updated (last update: 8 September 2026). Estimates above 16 hours exceed its reliable measurement range.
Protocol, configurations and original evidence →Task and measurement
Benchmark protocol profile
- Task / subject
- An online game supplies human attack/defense interactions; filtered subsets become separate extraction and hijacking benchmarks with explicit defense-validity checks.
- Measurement target
- How robust are instruction-following models against human-created prompt extraction and hijacking?
- Scoring / outcome
- Hijacking robustness rate · Hijacking robustness rate
- Interpretation boundary
- Adversarial filtering and reference-model selection shape the test population. Game success uses a narrow string criterion; it is not equivalent to real application compromise.
A reported evaluation
The source evaluates gpt-3.5-turbo-0613 · Prompt hijacking benchmark / Hijacking.
Reported measurement: Hijacking robustness rate · 18.4%.
Hijacking robustness rate Not disclosed
The study finds that apparently simple authorization instructions remain vulnerable and reports selected transfer demonstrations beyond the game.
Measurement dossier · NUWA reading
Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.
- Measurement construct
- Prompt hijacking/extraction robustness and defense validity.
- Task population
- A crowdsourced access-code game supplies adversarial attacks and defenses. Held-out evaluation tests model robustness against selected attacks.
- Scoring and failure modes
- HRR is the percentage of hijacking attacks that fail; ERR is the percentage of extraction attacks not revealing the exact access code. Defense validity measures correct access when the true code is supplied.
- System conditions
- Tables A1/A2 distinguish attacks and defenses filtered using the first three reference models; model snapshots are retained verbatim.
- Statistical meaning
- Not established by the retained material
- Validity and interpretation
- Higher robustness is better. Reference models participated in filtering, and defense validity must be interpreted jointly with robustness.
- Appropriate research use
- Not established by the retained material
- Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game · #S3; #S5; #S6; #S8fb1e829e9a35db2f01567c1b0e3bfcdbb34ff68c7e005bab5c99097c99991c35
Evidence across reports
The same benchmark name can cover different tasks, scoring rules and system configurations. Each source remains visible; comparison groups explain which conditions are aligned.
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Actual evaluators: Not disclosed
42 result records · 9 reported system names
Protocols and comparison decisions (0)
Original source charts and materials
Adversarial filtering and reference-model selection shape the test population. Game success uses a narrow string criterion; it is not equivalent to real application compromise.
Related research questions
References
Numerical results are reported by the cited authors.
- [1] Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game. 2023-11-02 ; #A3.T1; row gpt-3.5-turbo-0613; column Hijacking robustness rate; condition Hijacking; #A3.T1; row gpt-3.5-turbo-0613; column Defense validity; condition Hijacking