2026-09-11
Green Tests Are Not a Clean Bill of Health
An agent can make every test pass and still leave the codebase worse than it found it. Correct behavior is the entry fee. The factory also has to control what each change does to the system behind that behavior.
Correct is only one axis
Software can return the right answer and still be expensive to change. A new feature may pass its tests while duplicating an existing path, stretching one function across too many decisions, or adding an abstraction that exists only because the agent did not find the one already in the codebase. The release works today. The next change inherits a worse starting point.
This is not a special moral failure of AI-generated code. People make the same mess. Agents change the operating scale. They can produce more changes, across more surfaces, without the fatigue that used to slow the accumulation down. If your controls inspect only the result, structural debt can compound at the same speed as delivery.
That is why a green test suite is necessary and incomplete. Tests answer whether the behavior you specified still holds. They do not automatically answer whether the implementation duplicated a rule, concentrated too much logic in one place, or made tomorrow's work harder than today's needed to be.
Sloppiness needs signals
You cannot govern a factory with a reviewer saying, "This feels messy." The feeling may be right, but it arrives late, varies by reviewer, and gives the builder no stable boundary. The useful question is which observable changes tend to accompany the kind of mess you refuse to keep.
Start with plain signals. Did the change add far more code than the behavior appears to require? Did clone detection find a second copy of existing logic? Did complexity move into a small set of oversized functions? Did dependencies spread between modules that used to be separate? None of these measurements is a complete definition of quality. Together, they make deterioration visible before it becomes the codebase's normal shape.
The baseline matters more than a universal score. A mature service, a mobile application, and a disposable experiment should not share one arbitrary ceiling. Measure the repository you actually own, then stop ordinary changes from making its known weaknesses materially worse. A ratchet is useful because it asks a narrow question: did this run move the system in the wrong direction?
Use metrics as tripwires, not targets
The moment a proxy becomes the goal, the factory can satisfy the number while defeating the reason it exists. A hard line-count target rewards compressed code that nobody can read. A complexity ceiling can scatter one coherent operation across tiny functions and hide the coupling between them. A duplication score can encourage a premature abstraction that is harder to understand than two honest implementations.
So do not tell the agent to optimize the score. Use the score to trigger a closer route. When a change crosses a threshold, require an explanation, a smaller plan, a different implementation, or independent review against the actual design rule. The metric finds smoke. It does not get to declare where the fire is or how to put it out.
Good gates also carry escape hatches with evidence. Sometimes a larger change is the cleanest change. Sometimes duplication is temporary and deliberate. The exception should name the tradeoff, its owner, and the condition that removes it. An unexplained bypass teaches the factory that boundaries are optional. A recorded exception teaches it where judgment was applied.
Judge the change outside itself
The builder should not be the only authority on whether its work is clean. An agent that just chose an abstraction can produce a convincing defense of that abstraction. That defense is not proof. Give verification a separate context, the repository's actual constraints, and permission to reject a locally elegant answer that makes the larger system worse.
Mechanical checks should go first because they are cheap and consistent: formatting, types, tests, dependency rules, clone detection, complexity budgets, and change-size thresholds. Then use a focused evaluator for the questions those tools cannot settle. Does this code follow the established ownership boundary? Is the new concept real, or did the implementation invent it to make the task easier? Will the next feature have one place to change or three?
When verification finds a recurring failure, turn the finding into a control. Add the repository rule, sharpen the test, improve retrieval, or create a structural check. Sending the same review comment to the next agent is not learning. Making the next run inherit the correction is.
Throughput needs a shape
Agent-run delivery will produce more software with fewer people in the execution path. That advantage disappears if every fast change increases the cost of the one behind it. You do not have a factory when output rises while the machine quietly destroys its own ability to move. You have automated rework with a good demo.
Human judgment still decides what clean means for this system. People choose the architecture, the acceptable tradeoffs, and the moments when a metric should yield to context. But people should not patrol every line hoping to notice decay by instinct. Their decisions belong in baselines, gates, evaluation rubrics, and exception policies that run on every change.
Make behavior pass. Then make the structure hold. The software factory worth owning is not the one that generates the most code or closes the most tickets. It is the one that can keep changing the product without turning each new instruction into a negotiation with yesterday's shortcuts.
In response to Measuring the sloppiness of code by Earendil.