Skip to content
Neural Daily

ToolsNews

Google's Harness² tunes agent harnesses, not model weights

🇭🇺 Olvasd el magyarul →

Google Cloud AI Research has open-sourced Harness², a framework that leaves the model alone and improves the harness around it: prompts, memory, skills and scripts. A single step raised GLM-5's score on WorkBuddy-Bench by 11.5 points.

By: Mikola Máté Patrik · · 3 min read

Google Research diagram of the eight Harness² components around a frozen model
Image: Google Research · Press kit · Official project page diagram · Source

What a harness is

The harness is everything around the model: prompts, tools, memory, skills and orchestration. Harness² exposes eight editable components, each a file: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and a middleware plugin (arXiv). Model weights never change. The work comes from Google Cloud AI Research and Northwestern University.

How it works

The target is the open world: tasks are unseen in advance and there is no verifier to say whether an answer is right. Most self-improvement methods rely on labels or tests; Harness² works without them.

At task level it runs a Propose, Probe, Compose step. The agent first runs once with the base harness. An editor then proposes two candidates: one targets general agent failures such as planning or tool use, the other the knowledge and procedures of the current domain. Both run on the same task, and a composer keeps only the edits backed by observed behaviour.

Between tasks, a reflector stores the adopted harness with an EXPERIENCE.md file of lessons. This library guides future proposals, but each new task still starts from the base harness. A budget parameter k scales the search in parallel or sequential mode.

What the benchmarks show

The team tested three professional benchmarks: WorkBuddy-Bench, JobBench and Harvey's legal LAB benchmark. Executors were Gemini 3.5 Flash and GLM-5 under OpenCode and Codex, with Claude Opus 4.8 as the default editor. A single step improved all four setups on all three benchmarks:

  • WorkBuddy-Bench: GLM-5 under OpenCode rose from 64.2 to 75.7, overtaking Gemini 3.5 Flash on its base harness (72.4).
  • JobBench: gains of up to 4.5 points.
  • LAB: gains of up to 11.8 percentage points in per-criterion pass rate.
  • With parallel recursion at k=5, Gemini 3.5 Flash under OpenCode reached 80.8 on WorkBuddy-Bench.
Harness² benchmark chart: WorkBuddy-Bench scores versus base harness and Meta-Harness, and gains as budget k grows
Google Research · Press kit · Figure 1 of the Harness² paper · Source

Source: Google Research, Figure 1 of the Harness² paper. Left: WorkBuddy-Bench scores by domain (Gemini 3.5 Flash, OpenCode). Right: gains as the budget k grows.

Results also improved when the same model served as both executor and editor: all five models tested, from GLM-5 to Claude Opus 5, got better on every benchmark. On JobBench, the tool-error rate fell from 14.4% to 9.1%.

What it costs

It is not free: probing candidates means extra executions and editor sessions, which adds inference cost. The authors note that behavioural contrasts do not guarantee correctness, nor do they prove that an adopted edit transfers to a different task.

How to try it

The code is on GitHub under Apache-2.0 (GitHub), at version 0.1.0 (MarkTechPost). It needs Linux, Python 3.12 and uv. A local demo runs both recursion modes without model credentials; real runs call models through Vertex AI. OpenCode is the default executor, and Codex works with --substrate cx.

The paper's main message is practical: tuning the harness can be a cheaper win than switching models. If you build your own agents, version and measure skills, scripts and guardrails the same way you treat code.

    Produced with the help of AI, reviewed by an editor. How we make our articles

    Reviewed and edited by the Neural Daily desk. About

    Related articles