The Harness Must Not Grade Itself

Lilian Weng's survey of harness engineering lands on the part of agentic work that most software teams still miss. The model is one component. The surrounding system decides which tools it can touch, what context it sees, how work loops, where state survives, when another agent joins, and what evidence ends the run. Change that system and the same model behaves like a different worker.

That is why prompt tricks are already becoming a sideshow. A clever instruction can improve one attempt. A durable workflow can improve every attempt, retain the result, and route failure into the next run. Once the workflow itself becomes editable, the factory can mine repeated failures, propose a narrow change, test it, and keep the change when the evidence holds. That is real compounding leverage.

It is also where the governance problem stops being optional. A machine that can improve its operating instructions can improve the machinery that produces software. A machine that can improve the definition of passing can produce something much cheaper: confidence without improvement.

A self-improving harness needs two worlds. Inside the workshop are the surfaces an agent may change: routing rules, context selection, tool descriptions, retry behavior, delegation policy, and other bounded parts of execution. Outside it are the authorities that decide whether the change worked: permissions, evaluation data, consequence thresholds, audit records, and the mechanism that accepts or rejects the candidate.

If the proposer can edit both worlds, every gain is suspect. It can remove a difficult check, choose friendlier tasks, increase its budget, hide a failure, or redefine an incomplete artifact as complete. None of those moves requires malice or awareness. They are ordinary ways to satisfy an objective when the boundary permits them. Optimization follows the score you built, including the shortcuts you forgot to close.

Make the separation structural. Give the evolving harness write access only to named files or typed configuration fields. Keep the evaluator in a different permission domain. Record every candidate as a diff. Run it against evidence the proposer cannot alter or preview completely. The evaluator should return a decision, not an invitation to negotiate the meaning of the test.

Failure mining creates a powerful temptation. The system finds a recurring mistake, writes a targeted rule, and passes the cases that used to fail. That proves the patch learned those cases. It does not prove the harness became better. The new rule may bloat context, block a valid path, overfit one repository, or trade a visible failure for a quieter one somewhere else.

Every proposal needs two burdens of proof. First, reproduce the weakness and show that the candidate fixes it. Second, show that behavior worth preserving still survives. The preservation set must include more than yesterday's happy path. It should cover different task shapes, risky boundaries, and cases where the correct behavior is to stop rather than finish.

Do not merge several ideas into one candidate and celebrate the aggregate score. Small edits make causality legible. Attach the failure pattern, predicted effect, at-risk behavior, evaluation result, and final decision to each one. Rejected changes belong in the record too. Otherwise the factory repeats dead experiments and calls the rediscovery progress.

Agent systems produce an ocean of traces. Saving all of it is not the same as learning. Raw logs become another context window nobody can afford to read, and summaries can erase the exact transition that caused the failure. Useful observability connects a consequence to a component: what failed, which behavior caused it, which harness surface could change it, and what result would falsify the proposed fix.

That chain turns self-improvement from reflection theater into engineering. A timeout may come from wasteful planning, a broken tool, a deadlocked delegation pattern, or a task that should have been rejected at intake. Treating the terminal symptom as the cause produces rules that merely move the failure around. The factory needs enough durable evidence to trace the mechanism, then a compact claim the next evaluation can attack.

Human judgment still owns the outer decision. People choose which outcomes matter, what risk is unacceptable, and which tradeoffs the score must not collapse into one number. Agents can search the design space faster than people can. They should. But speed inside the workshop does not transfer ownership of the building code.

Our position is that self-improving harnesses will separate serious software factories from fleets of disposable agents. The organizations that win will encode operating experience into machinery instead of asking each new run to rediscover it. Their factories will get better at routing work, preserving context, recovering from failure, and producing evidence without waiting for a person to rewrite the playbook.

The advantage will not come from allowing unlimited self-modification. It will come from a narrow loop with teeth: observe a failure, name its mechanism, propose a bounded change, test the claimed repair, attack the behaviors it might damage, and accept nothing without independent proof. Most candidates should be allowed to lose. A loop that cannot reject its own idea is not learning. It is accumulating instructions.

Start by drawing the boundary before you automate the improvement. Name what the agent may edit. Put permissions, evaluators, acceptance rules, and audit history outside that surface. Make every claimed gain survive conditions the proposer did not choose. Then let the factory work on itself. The machinery can evolve quickly once the right to declare victory stays somewhere it cannot reach.