Risk Evaluation · Method

Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations

NUWA research note · independent review pending

First published: 2026-10-07Source checked: 2026-10-25

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: navigable provenance helps inspect a behavioral claim without itself proving the label correct.

Research question

How can evaluators preserve observability and auditability in very long agent runs?

Design and assumptions

Transect separates reusable task/behavior specifications from judge settings and aligns events, token use, subagents and labels on a source-linked timeline.

Source findings

The case study analyzes an AI R&D evaluation producing almost thirteen million tokens.

The conclusion distinguishes reliability diagnostics from validity, which requires separate evidence such as independent expert annotations.

NUWA interpretation

NUWA interpretation: navigable provenance helps inspect a behavioral claim without itself proving the label correct.

Limits and contrary evidence

LLM scanners can overstate confidence and obscure uncertainty; the case study is not a validation across all scientific work.

Research and decision relevance

Model recommendation: preserve source-turn anchors, judge configuration and disagreement diagnostics with exported analyses.

Precise source locations

Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations · AbstractSelected original resource https://arxiv.org/html/2610.08364; sha256:15cf35ee7c1d3115d5dac5322b3352f66d9b2564d843dfac2b42e43cf09c6480; archived source capture; exact capture time not independently established
Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations · §2.2 Design principlesSelected original resource https://arxiv.org/html/2610.08364; sha256:15cf35ee7c1d3115d5dac5322b3352f66d9b2564d843dfac2b42e43cf09c6480; archived source capture; exact capture time not independently established
Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations · §3 Case Study: Reading an AI R&D Evaluation with TransectSelected original resource https://arxiv.org/html/2610.08364; sha256:15cf35ee7c1d3115d5dac5322b3352f66d9b2564d843dfac2b42e43cf09c6480; archived source capture; exact capture time not independently established
Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations · §4 ConclusionSelected original resource https://arxiv.org/html/2610.08364; sha256:15cf35ee7c1d3115d5dac5322b3352f66d9b2564d843dfac2b42e43cf09c6480; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Published by UK AI Security Institute

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{transect,
  title = {Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations},
  url = {https://www.aisi.gov.uk/research/transect-retaining-observability-for-long-horizon-llm-agent-evaluations},
  year = {2026},
  date = {2026-10-07},
  version = {Source capture 2026-10-25},
  note = {Source capture 2026-10-25}
}

NUWA record · BibTeX

@misc{transect-nuwa,
  title = {Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/transect/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--transect.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--transect.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Transect: Retaining Observability for Long-Horizon LLM Agent Evaluations. 2026-10-07 ; Official publication masthead and article byline; Abstract; §2.2 Design principles
[2] Read the full paper