Harness engineering: the runtime is the product

Companion code: 11-harness

TL;DR

  • The harness is everything around the model: the loop, durable state, tools, sandboxing, verification, budgets and traces. In 2026, it’s where most of the difference between a demo agent and a dependable one lives.
  • Teams running agents at scale have converged on similar practices. Make the work explicit. Keep progress outside the model. Give each chunk of work a fresh context. Let verification define “done”. Encode standards as checks, not instructions. And keep the “brain” separate from the “hands”.
  • Build harnesses to outlast the model they were written for: what compensates for a weakness today becomes dead weight when the next model fixes it.

From prompts to context to harnesses

There’s a common way to tell the recent history of building with language models:

  • 2022–24, prompt engineering: get the instructions right.
  • 2025, context engineering: get everything the model sees right (post #3).
  • 2026, harness engineering: get everything around the model right.

The shift happened because models became capable of long, autonomous work: hours of coding, research or operations across many context windows. At that length the model isn’t the main source of failure. What fails is the system around it: state lost between sessions, work declared finished that wasn’t, a crash that loses an hour of progress, an agent that wanders off course with no check to stop it.

Post #1 introduced the harness as “everything that isn’t the model call”. This post is about what it takes when the runs get long.

What a harness owns

The model sits inside a harness. The harness runs the agent loop and owns everything durable: a work list or progress file, an append-only session log, checkpoints, and the verification gate that decides when work is done. Tools run in a separate sandbox, the hands, reached through one execute interface, and credentials stay outside the sandbox. Policy checks, budgets and traces wrap the whole thing.
The model decides. The harness remembers, executes, checks and limits.

Seven principles from the field

The principles below come from four write-ups by teams running long-horizon agents. Two are from Anthropic: Effective harnesses for long-running agents (November 2025) and Scaling Managed Agents (April 2026). One is OpenAI’s Harness engineering (February 2026), in which a small team spent five months building a product of about a million lines of code without writing any of it by hand (InfoQ’s summary covers the details). The fourth is the parallel compiler project from post #7.

1. Make the work explicit before anyone starts

Anthropic’s long-running coding setup begins with an initializer agent whose only job is to turn a vague request into a detailed list of features, each marked pass or fail, plus a script to start the environment and a first commit. Every later session works from that list. Without it, they saw two opposite failures: agents trying to build everything in one go, and agents declaring the project finished halfway through. A written work list fixes both, because “done” means “every item passes”.

2. Keep progress outside the model

Each new session starts with no memory of the last one. So the harness keeps the record: a progress file, the git history, a log of what was tried. Anthropic’s Managed Agents design makes this formal. The session is an append-only event log that lives outside any context window. The model can query slices of it, and a crashed harness simply resumes from the last event. The context window becomes disposable, and the log is the source of truth.

3. One bounded chunk of work per context

Rather than one heroic context that grows for hours, a long run becomes a sequence of short ones: read the progress notes, take the next item, finish it, verify it, commit, record the result, end. This is post #3’s compaction idea, turned into an architecture. Each item gets a clean context, and the work list plus the log carry continuity between items.

4. Let verification define “done”

Every one of these write-ups comes back to verification. Anthropic’s agents could mark a feature complete only after testing it end to end in a real browser, as a user would. The compiler team’s main lesson was to write extremely good tests, because an autonomous agent solves whatever problem the tests define. OpenAI’s agents checked their own changes against the application’s logs, metrics and traces. The harness, not the model, decides when work is done, and it decides with checks that can fail.

5. Encode standards as checks, not paragraphs

OpenAI’s team didn’t rely on prose instructions to keep a million-line codebase coherent. They enforced architecture mechanically, with custom linters and structural tests. For example, code dependencies had to run in one direction through fixed layers, from types and configuration down to the user interface, and a violation failed the build. Documentation kept in the repository (maps, execution plans and design specifications) served as the agents’ system of record. An instruction in a prompt is a suggestion; a failing check is a fact.

Research is starting to measure how teams do this. A 2026 study of how developers configure coding agents (Galster et al.) examined 2,853 GitHub repositories. Plain context files dominate, with AGENTS.md emerging as a standard that works across tools, and few teams yet use more powerful mechanisms such as packaged skills or sub-agents. Most harnesses are still mostly text.

6. Separate the brain from the hands

Anthropic’s Managed Agents post describes splitting the brain (the model and its loop) from the hands (the sandboxes where tools and generated code run). They meet at a single, simple interface: run this tool with this input, and get a string back. The payoffs are practical:

  • Either side can fail and be replaced on its own.
  • Sandboxes are created only when a tool actually needs one, which cut time-to-first-token by about 60% at the median and over 90% at the 95th percentile.
  • Credentials never enter the sandbox where model-written code runs, which removes a whole class of the attacks from post #10.

7. Don’t overfit the harness to today’s model

Harnesses accumulate workarounds for model weaknesses. The Managed Agents post gives an example: special handling for a model’s tendency to wrap up work early as its context filled, which turned out to be unnecessary for the next model. A harness built around stable interfaces (durable sessions, swappable sandboxes) and external checks survives model upgrades. One built around compensating for one model’s quirks has to be rewritten each time. The 2026 OneDayAgent paper points the same way: its harness ran unchanged across five backend models from three families.

The experiment

The companion code (11-harness) is a small harness of about 170 lines that applies these principles to a modest job: summarizing incident reports one at a time.

  • Explicit work list. An initializer step writes progress.json with every item and its status before any agent runs.
  • Durable state. Every model call, tool call and verification is appended to events.jsonl. The progress file is written atomically, so a crash never leaves it half-written.
  • One context per item. Each report gets a fresh agent conversation; nothing grows across items.
  • A verification gate in code. A summary counts as done only if it names the right service, the right number of minutes and the right root cause, in at most 35 words. A failure goes back to the agent with the reason:
problems = [p for p, ok in [
    (f"must name the service '{service}'", service in summary.lower()),
    (f"must give the impact ({minutes} minutes)", minutes in summary),
    (f"must give the root cause '{cause}'", cause in summary.lower()),
    ("must be one sentence of at most 35 words", len(summary.split()) <= 35),
] if not ok]
  • Bounded effort. Each item gets a step limit and three attempts, then it’s flagged for a human instead of looping forever.
  • One interface to the hands. Every tool call goes through execute(name, args), so the tools could move into a container or a remote service without touching the loop.

And you can crash it:

python harness.py --limit 8 --crash-after 3     # stops after 3 verified items
python harness.py --limit 8                     # resumes at item 4; nothing is redone

In a test run with a scripted stand-in model, one summary left out the minutes. The gate rejected it, the retry received the reason and passed, and after the simulated crash the second run picked up exactly where the first had stopped. The event log recorded the whole history across both runs.

Trade-offs

  • Harnesses add machinery. A progress file, an event log, a gate and a retry policy are more code than a bare loop. For a five-minute task, a bare loop is fine. The machinery pays off when runs are long, expensive or hard to restart.
  • Verification is the hard part. The gate above is easy because the facts are checkable. Many real tasks need tests, a browser, or a model-based judge with a rubric. Posts #12 and #13 are about measuring that well.
  • Mechanical rules can over-constrain. Linters and structural tests encode today’s idea of good architecture. Review them like code, or the harness will enforce yesterday’s decisions forever.

Next in the series: Evals for agents: pass@k is lying to you.

Sources & further reading

#harness #long-running-agents #architecture