2026-08-04

The Skill Issue Is the System

If your coding agent keeps producing confident nonsense, stop arguing about whether the model can program. Inspect the machine you put around it.

A chat window is not a software factory

Ask a general-purpose model to enter an unfamiliar repository, infer the local architecture, choose the right implementation, and judge its own result. It may overengineer a small change. It may call an API that feels plausible but does not exist. It may attack the visible symptom for hours while the actual fault sits in an asset, configuration, or boundary it never examined. Those failures are real. The conclusion that agents cannot build software is not.
The setup assigned one probabilistic component every role in the system. It had to retrieve context, decide what mattered, plan the change, implement it, select the proof, and declare itself finished. A capable engineer would struggle under the same operating model. We would call the environment chaotic, not publish a theory that humans cannot program.
This is the uncomfortable response to Mathieu Ropert's account of AI programming: the failure mode is mostly a skills issue. Not prompt cleverness. Factory-building skill. The model is a productive asset only after the surrounding system constrains what it may assume, exposes the knowledge it needs, and refuses work that has not survived independent proof.

Context must be routed, not dumped

A repository contains more information than any single run should consume. The answer is not to paste more files into a conversation and hope attention behaves like understanding. Give the factory retrieval routes: architecture decisions near the code they govern, stable commands for common operations, searchable failure history, ownership boundaries, and explicit rules for which sources outrank which summaries.
Then make provenance part of the route. Internal search can uncover the discussion that explains a strange design choice, but a summary of a summary is not evidence. Work in progress must not certify itself. The retrieval stage should preserve links to the underlying artifact, distinguish decided policy from speculation, and let the verifier inspect the source before a claim becomes a change.
That is how you prevent the telephone game without giving up the speed of machine search. Agents can explore many possible trails. The factory decides which kinds of trail are admissible and carries the origin forward so later stages do not mistake repetition for truth.

Plausible defaults need hard boundaries

Models are very good at producing the shape of familiar software. That is exactly why an unnecessary abstraction or imaginary method can look reasonable in review. If a request says only what to change, the agent fills in how from patterns learned elsewhere. Sometimes that is useful. Sometimes it imports a design the repository has already rejected.
A factory narrows that freedom before implementation. The specification names the observable outcome, the parts that may change, the constraints that must remain true, and the evidence required at the exit. Repository rules ban known-bad methods mechanically. Existing patterns are discoverable through code, not folklore. When two plausible choices carry different consequences, the line stops and routes the decision to its owner.
Human judgment belongs at that boundary. People choose the consequence and decide which risks are acceptable. Agents perform the repeatable transformations inside it. That does not preserve every coding role. It moves valuable judgment higher and removes the typing, searching, checking, and coordination that machinery can execute more consistently.

The builder cannot grade itself

Telling an agent to double-check its answer is ceremony. The same context and assumptions that produced a wrong implementation will often produce an elegant defense of it. Proof has to come from somewhere the builder did not control: executable acceptance criteria, static restrictions, focused tests, runtime observation, and a separate verification stage encouraged to break the result.
The proof must also reach the real seam. A textbook fix can pass a convenient unit test while failing in the shipping runtime. A code change can look correct while the defect lives in data. A generated answer can cite a document that merely repeats the answer. Good gates test the consequence under conditions that expose those substitutions.
When a failure escapes, do not just correct the current output. Encode what the factory should learn. Add the missing test, retrieval rule, runtime probe, or ownership gate. The next agent should inherit the correction without needing the engineer who discovered it to stay in the loop forever.

Judge the operating system, not the demo

A weak agent demo and a weak agent critique share the same mistake: both treat one model interaction as the product. The optimist shows a clean prompt producing code and declares engineering solved. The skeptic shows a bad run producing the wrong abstraction and declares engineering irreplaceable. Neither has measured an operating system.
Our position is that agents will replace repeatable software-delivery work, but not because every raw answer is brilliant. They will replace it because factories can route context, constrain authority, generate many attempts, reject bad ones mechanically, and retain every correction. Human teams lose process when people leave, forget, or improvise. A governed factory compounds it.
So when the agent fails, ask the useful question: what allowed this failure to reach a person? If the answer is missing context, an undecided requirement, unrestricted authority, or self-selected proof, improve the machinery. Blaming the model may feel like expertise. Building the system that makes its weaknesses survivable is the expertise that changes the business.
In response to An Honest Review of AI Programming by Hacker News.