How long can an agent work? Time horizons explained

Companion code: 13-time-horizons

TL;DR

  • An agent’s 50% time horizon is the length of task, measured by how long a skilled human takes, that the agent completes successfully half the time. It turns benchmark scores into something intuitive: “it can do tasks that take a person about an hour.”
  • METR’s measurements show the horizon of frontier agents doubling every few months, and faster recently. A January 2026 update put the doubling time since 2024 at about 89 days.
  • It’s not “how long an agent can run”, it’s measured at 50% reliability, it comes from mostly software tasks, and its error bars are wide. Use it to track trends; use your own evals to make decisions.

Turning scores into time

Benchmark scores are hard to interpret. Does going from 62% to 71% on some suite mean your agent can now handle a morning’s work, or only a slightly harder ten-minute job? In March 2025, METR (Model Evaluation & Threat Research) proposed a way to answer that: Measuring AI Ability to Complete Long Software Tasks (Kwa, West, Becker et al.).

The method has three steps:

  1. Time people. Skilled humans attempt a large, varied set of software and research tasks, from a few seconds to many hours long, and their completion times are recorded.
  2. Run the agent on the same tasks and record which it completes.
  3. Fit a curve. Plot the agent’s success against how long each task took a human, on a log scale, and fit a logistic curve. The task length where the curve crosses 50% is the 50% time horizon; where it crosses 80%, the 80% horizon.
Success probability plotted against human task length on a log scale from one minute to sixteen hours. The curve falls from near 100 percent for short tasks to near zero for long ones. The 80 percent time horizon is where it crosses 80 percent, about 21 minutes; the 50 percent horizon is where it crosses 50 percent, 60 minutes, for this synthetic agent.
How a horizon is read off a fitted success curve. A synthetic agent from the companion code, not a real model.

The headline finding was the trend. Over 2019–2025, the 50% horizon of the best available agents had doubled about every seven months, and the authors noted it might have sped up in 2024. In early 2025 the frontier sat at around an hour.

How fast it’s moving

In January 2026 METR published Time Horizon 1.1, with a larger task suite (228 tasks, up from 170) and new evaluation infrastructure. Re-estimated on that basis, the doubling time since 2023 came out at about 131 days, and since 2024 at about 89 days.

Doubling times are hard to feel, so here they are as yearly growth:

Doubling every Growth per year
~7 months (the original 2019–2025 trend) ≈ 3.3×
131 days (since 2023, TH 1.1) ≈ 6.9×
89 days (since 2024, TH 1.1) ≈ 17×

If it holds, the 89-day pace takes an hour-long horizon to a working day within about a year. That “if” is doing a lot of work, which brings us to what the number does not say.

What the horizon is not

METR published a note on exactly this, Clarifying limitations of time horizon (January 2026). The main points, paraphrased:

  • It’s not how long an agent can run. A 4-hour horizon doesn’t mean an agent works unattended for 4 hours. It means the agent completes, half the time, tasks that take a person about 4 hours. The agent might finish in minutes, or churn for much longer.
  • 50% is not reliable. A horizon at 50% success says nothing about tasks you need done right 98% of the time. The 80% horizon is much shorter; in the synthetic example above, it’s about a third of the 50% horizon. Post #12’s point about pass^k applies with full force.
  • It depends on the domain. The suite is mostly software and research tasks. METR notes horizons vary by orders of magnitude between domains, and are 40–100× lower for visual computer-use tasks.
  • The error bars are wide. Historically about 2× in each direction. METR’s own example is Claude Opus 4.5, whose 50% horizon confidence interval runs from 1 hour 49 minutes to 20 hours 25 minutes. METR’s results page also notes that measurements above roughly 16 hours aren’t reliable with the current tasks.
  • Doubling the horizon doesn’t double automation. Real jobs are mixtures of tasks of many lengths, with context, coordination and judgment that benchmark tasks lack.

None of this makes the metric bad. It makes it a trend indicator: the best single number we have for how fast agent capability is growing, not a spec sheet for your use case.

Other ways to ask “can it do real work?”

Two other approaches are worth knowing, because they measure value instead of task length:

  • GDPval (Patwardhan et al., OpenAI, 2025) builds tasks from the real work of experienced professionals in 44 occupations across the sectors that contribute most to US GDP, and compares model output with expert output. Its authors found frontier models approaching expert quality on many deliverables, and steadily improving.
  • Agents’ Last Exam (Sun et al., June 2026) collects more than 1,000 long-horizon professional workflows across 55 subfields, built with over 250 industry experts, each with a verifiable outcome. On its hardest tier, the average full pass rate across current setups is below 1%. Horizon charts that look steep can coexist with very hard, unsaturated benchmarks of real work.

Estimate a horizon for your own agent

You can apply the same method to your own eval results. You need two things per task: roughly how long a skilled person takes, and whether your agent succeeded. The companion script (13-time-horizons) fits the logistic curve and bootstraps the uncertainty. It needs no API key and no third-party packages.

To check that the method works, the script first simulates an agent with a known 50% horizon of 60 minutes and tries to recover it from noisy pass/fail data. Here’s what it reports for different numbers of tasks (synthetic data, fixed seed, actual output of the script):

Tasks Estimated 50% horizon 90% interval
30 65 min 27 min – 2.3 h
150 54 min 38 – 74 min
1,000 65 min 58 – 73 min

With 30 tasks, the honest answer is “somewhere between half an hour and over two hours”. It takes hundreds of tasks to pin a horizon down. That’s one reason METR’s suite has over two hundred, and why the horizon of your agent on your 20-task eval should be reported with its interval, if at all.

The fit itself is a few lines of Newton’s method for logistic regression on log task length:

def horizons(a: float, b: float) -> tuple[float, float]:
    """Return the 50% and 80% horizons in minutes. b must be negative (longer tasks are harder)."""
    return 2 ** (-a / b), 2 ** ((math.log(4) - a) / b)

Trade-offs

  • Human time estimates are the weak link. Guessed durations are usually too optimistic. Time real people on a sample, and use the same convention throughout. METR notes that its own conventions for human baselines shift estimates by more than 1.25×.
  • Short suites can’t show a trend. If all your tasks take five to twenty minutes, there’s no slope to fit. Include a spread of task lengths, or report plain success rates instead.
  • A horizon hides the shape. Two agents with the same 50% horizon can have very different 80% horizons. For anything users rely on, report the 80% one too.

Next in the series: Agents that design agents: ADAS → DGM → AlphaEvolve.

Sources & further reading

#evals #measurement #long-horizon