Risk Evaluation · Method

Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT

NUWA research note · independent review pending

First published: 2026-07-21Source checked: 2026-10-25

Original source ↗Cite this record ↓Autonomy →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: this cost-based illustration addresses optimization economics under explicit labor assumptions.

Research question

At what expenditure does autonomous optimization match estimated human optimization?

Design and assumptions

NanoGPT speedrun results are modeled against monetary effort; human estimates combine contributor interviews and an LLM judge, alongside agent compute/token costs.

Source findings

After more than $10K agent expenditure, the illustration estimates expenditure horizons of $0–$3K.

The metric is the crossing of estimated cost–performance curves, not elapsed-time autonomy.

NUWA interpretation

NUWA interpretation: this cost-based illustration addresses optimization economics under explicit labor assumptions.

Limits and contrary evidence

Smooth returns, historical human-effort estimates and contamination affect the crossing; hybrid human–AI work is outside the metric.

Research and decision relevance

Model recommendation: report human-cost assumptions and curve uncertainty instead of converting the horizon into autonomous workdays.

Precise source locations

Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT · IntroductionSelected original resource https://metr.org/blog/2026-07-21-expenditure-horizon/; sha256:b852697d512cce8f3e1ab1a4f879449bd3d7e520792e4a2f371bbfd56f524bd4; archived source capture; exact capture time not independently established
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT · Estimating the Returns to Human Labor on NanoGPTSelected original resource https://metr.org/blog/2026-07-21-expenditure-horizon/; sha256:b852697d512cce8f3e1ab1a4f879449bd3d7e520792e4a2f371bbfd56f524bd4; archived source capture; exact capture time not independently established
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT · Estimating agent expenditure horizonsSelected original resource https://metr.org/blog/2026-07-21-expenditure-horizon/; sha256:b852697d512cce8f3e1ab1a4f879449bd3d7e520792e4a2f371bbfd56f524bd4; archived source capture; exact capture time not independently established
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT · Introduction: highlighted limitationsSelected original resource https://metr.org/blog/2026-07-21-expenditure-horizon/; sha256:b852697d512cce8f3e1ab1a4f879449bd3d7e520792e4a2f371bbfd56f524bd4; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Published by METR

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{expenditure-horizon,
  title = {Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT},
  url = {https://metr.org/blog/2026-07-21-expenditure-horizon/},
  year = {2026},
  date = {2026-07-21},
  version = {Source capture 2026-10-25},
  note = {Source capture 2026-10-25}
}

NUWA record · BibTeX

@misc{expenditure-horizon-nuwa,
  title = {Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/expenditure-horizon/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--expenditure-horizon.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--expenditure-horizon.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT. 2026-07-21 ; Official publication masthead and article byline; Introduction; Estimating the Returns to Human Labor on NanoGPT