Harness2: Label-Free Agent Harness Tuning, Explained

Harness2 is a new open source framework from Google Research that improves how an AI agent performs by tuning everything around the model instead of the model itself. The repository appeared on GitHub on October 6, 2026, with a paper on arXiv three days later, and it has already passed 12,000 stars. The pitch is simple: most teams cannot retrain the model they use, but they can restructure how that model is prompted, tooled, and checked. Harness2 automates exactly that layer. Source Source

This post explains what Harness2 actually does, what the published results show, and what it means if you run agents in production. Primary sources first: the GitHub repository and the arXiv paper. Anything that is our read of the material is marked as such.

What Harness2 is

The paper defines a harness as the system around a model that lets it act as an agent: prompts, tools, memory, skills, orchestration, and more. "Harnessing" is the process of optimizing that system. The core idea is expressed as Agent equals Model plus Harness, and Harness2 only ever edits the second term. Model weights stay frozen throughout. Source

The framework targets an "open world" setting, meaning agents meet tasks that were unseen when the harness was designed. That creates two problems the paper names directly: there are no ground-truth verifiers to guide edits at test time, and improvements tuned for one task often fail to generalize to the next. Harness2 addresses both with two connected levels of recursion, which we unpack below. Source

The release is plug-and-play under an Apache 2.0 license. It supports parallel and sequential recursion modes, runs on OpenCode and Codex executors, and ships with three professional benchmarks: JobBench for professional work, LAB for legal work, and WorkBuddy-Bench for office, web, and code tasks. One caveat in the fine print: this is not an officially supported Google product. Source

The eight components it can edit

Harness2 exposes eight editable components around the model: the system prompt, guardrails, memory, skills, tool guidance, scripts, subagents, and a middleware plugin. Each component is mounted as a file in the runtime, which means every harness edit is a concrete, inspectable file change rather than an abstract configuration tweak. Source

That decomposition matters for anyone running agents today. These eight files are exactly the things teams already hand-edit when an agent misbehaves: the prompt, the tool descriptions, the memory policy, the delegation rules. Harness2's contribution is to make those edits systematic, evidence-based, and reversible instead of vibes-based. Source

Propose, Probe, Compose: tuning without labels

The hard part of improving an agent on a new task is that you usually cannot tell whether the output was correct. Harness2's answer is a contrastive step called Propose-Probe-Compose. It starts by running the current harness once to get a reference execution. The point of that run is not to judge correctness but to observe behavior: where the agent hesitated, which tools it called, how it managed context. Source

Propose then drafts two candidate edits. The "invariant" candidate targets recurring agentic weaknesses such as planning, tool use, context management, or delegation. The "domain-specific" candidate targets the knowledge and constraints of the current task's domain. Both candidates are full executable harnesses built by editing the component files. Source

Probe runs the candidates under matched conditions and compares their behavior against the reference. Compose then builds a successor harness from the changes that the behavioral evidence supports, keeping parts of both candidates when each earned its place. No ground-truth answer is consulted at any point; the loop runs entirely on observed execution evidence. Source

Two scaling modes sit on top of this step. Parallel recursion explores multiple candidate harnesses at once and composes from their combined evidence. Sequential recursion carries the refined harness through several successive Propose-Probe-Compose steps. In the repo's framing, parallel explores wider while sequential refines deeper. Source

The second recursion: a memory for improvement itself

Most tuning loops are amnesiac: each task starts from scratch. Harness2's second level of recursion fixes that by maintaining an improvement harness across tasks. After each task, the pipeline retains the adopted harness plus the lessons from its refinement, so the next task's proposals start from accumulated evidence rather than zero. Source

Each entry in this library pairs an executable harness with an EXPERIENCE.md file that records the applicable situation, the lessons, and the supporting evidence. An index lets later sessions pull only relevant entries. Notably, when the evidence is too thin, reflection adds nothing, which keeps the library from filling with noise. Source

The ablation results argue this memory is doing real work. Removing the library drops scores by 3.1 points on WorkBuddy-Bench and 1.8 on JobBench. Removing the invariant-versus-domain scoping costs 2.9 and 3.7 points, and removing the composition step costs 1.6 and 1.9. All three design choices contribute measurably. Source

What the results show

Across three professional benchmarks and four model-and-harness configurations, a single Harness2 step consistently improved GLM-5 and Gemini 3.5 Flash over their base harnesses. The reported gains reach 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB's partial pass rate. Source

The most quotable result: on WorkBuddy-Bench, one step lifts GLM-5 from 64.2 to 75.7 under OpenCode and 75.3 under Codex, surpassing the base Gemini 3.5 Flash scores of 72.4 and 74.5. A weaker model with a tuned harness beat a stronger model running its original harness. That is the whole thesis in one number. Source

Spending more recursion budget keeps helping, with diminishing shape. With Gemini 3.5 Flash under OpenCode, raising the task-level budget k from 1 to 5 lifts gains over the base harness from 4.5 to 7.5 points on JobBench and from 5.1 to 8.4 on WorkBuddy-Bench. The paper reports both tested Harness2 configurations sit on the empirical compute-performance Pareto frontier against baselines like ReasoningBank and Meta-Harness. Source

Why this matters for teams running agents

Here is our read, separate from the paper's claims. Most production agent teams live exactly where Harness2 operates: the model is a fixed API dependency, and all the leverage sits in the scaffolding. Until now, harness edits were shipped on intuition and reverted after incidents. Harness2 encodes a discipline instead: every change must earn its place through measured behavior on real tasks, and the reason it worked gets written down where the next task can use it.

That discipline pairs naturally with observability. If you cannot see what your agent did, you cannot judge a harness change, which is the same premise behind our AI agent observability checklist. It also rhymes with repeatable evaluation practice, the theme of our post on Scale's AgentEnv and repeatable agent evaluations. Harness2 adds the missing third piece: acting on that evidence automatically.

The practical implication for AgentOps is that "harness version" becomes a first-class release artifact. The Harness2 results directory already treats adopted harnesses, reflection notes, and evaluation results as saved outputs per run. Expect teams to start reviewing harness diffs the way they review model version bumps today. Source

Trying it yourself

Setup is Linux, Python 3.12, and uv. From the repository root you create a virtual environment, install the package, and run the demo, which exercises both recursion modes with deterministic editors. A --dry-run flag previews task selection without model calls, which is the sensible first step before spending API budget. Source

Benchmark runs follow a five-command pattern per benchmark: setup, check, refine, judge, report. Judging and reporting must use the same benchmark, executor, subset, recursion budget, mode, and seed as the refinement run, or the numbers will not line up. Results land under a runs directory organized by benchmark, including task summaries, deliverables, adopted harnesses, reflection notes, and evaluation scores. Source

One security note deserves emphasis. The executor's network access is unrestricted and Codex runs with its sandbox disabled by default, and public benchmark rubrics may be reachable online. The README explicitly tells you to configure network restrictions before evaluation so agents cannot reach benchmark answers or grading criteria. Treat that step as mandatory, not optional. Source

Limits and open questions

The evidence is strong but narrow in specific ways. The gains are demonstrated on three professional benchmarks with two models and two executors. Single-step gains range from 11.5 points down to 1.2, so the effect size varies considerably by configuration. The repo also warns that benchmark runs carry API costs and recommends starting with a small task selection. Source Source

Open questions we would want answered before betting a production stack on it: how the loop behaves on messy internal tasks rather than curated benchmarks, how often the composed harness regresses on the task it was tuned for, and what the ongoing token cost looks like at k=5 compared to just buying a stronger model. The compute-performance analysis is a good start, but every team's cost curve is different. Source

Bottom line

Harness2 turns agent tuning into an evidence loop: propose an edit, probe its behavior, keep only what works, and remember why. The headline finding, that a weaker model with a tuned harness can beat a stronger model on its stock harness, gives every team stuck on a fixed model a concrete new lever. It is early, benchmark-bound, and unofficial, but the loop it encodes is a reasonable default for maintaining an agent stack in 2026.