Harness-Zero: Agent-as-Harness Distillation for LLM Agents

What Happened

ArXiv preprint Harness-Zero introduces harness distillation, a method that replaces hand-engineered agent scaffolding with an agent trained to act as the harness itself. The paper appeared simultaneously on ArXiv and the Hugging Face papers feed, indicating early distribution through both the research and practitioner channels. The core claim is that harness behavior — the tool-routing, retry, memory, and control logic conventionally written in code around a base model — can be learned rather than authored.

Why It Matters

Harness engineering is currently the dominant hidden cost in production agent pipelines: each task family accumulates bespoke scaffolding, and that scaffolding does not transfer cleanly across models, tools, or task distributions. Distillation reframes the harness as a learned artifact, which means deployments can carry a compact policy for orchestration instead of a codebase of brittle heuristics. Builders running multi-task or multi-tenant agents benefit most, since the marginal cost of standing up a new task shifts from engineering hours to a distillation run. For operators, the strategic implication is that harness quality becomes a training-data and evaluation problem rather than an integration problem — a different budget line and a different failure mode.

Technical Details

The method treats the harness as a policy and supervises it with trajectories collected from an existing agent-as-harness configuration, then distills that behavior into a smaller or cheaper executor. Reported gains center on task success parity with the teacher at reduced per-task scaffolding and lower inference-side orchestration overhead; the paper places it in the lineage of agent distillation work (e.g., FireAct-style trajectory imitation) while shifting the distillation target from the model to the surrounding control loop. It sits on ArXiv and Hugging Face, with no disclosed production benchmark suite at the time of writing. Integration requirements mirror existing agent distillation: a teacher agent capable of producing successful trajectories, a scoring or filtering step to remove low-quality rollouts, and an evaluation harness that captures tool-use correctness rather than final-answer accuracy alone. Known limitations include teacher quality dependence, distribution shift when the executor model differs from the teacher, and unclear robustness when the environment's tool APIs change post-distillation.

Operational Impact

Teams gain a pathway to replace per-task scaffolding code with a single distilled orchestration policy, reducing the surface area that must be re-tested when the underlying model or tool schema changes. Evaluation burden shifts: instead of unit-testing harness branches, operators must curate trajectory datasets and maintain offline replay for the distilled policy. Cost structure changes from mostly-engineering to mostly-collection-and-compute, which favors teams already running high-volume agent traffic that can supply teacher trajectories. Existing hand-authored harnesses are not obsolete, but they become candidates for distillation once a task family stabilizes. The immediate workflow change is a new step in the agent release pipeline — distillation run, offline replay, promotion — sitting between prompt iteration and canary traffic.