An Experiment Can Win Without a Release

An experiment promises evidence that resolves uncertainty; a delivery task promises working behavior under agreed conditions. A prototype is a temporary implementation built to investigate a question. Its existence alone proves neither that the question was answered nor that the software is ready for users.

Christian Bird's talk page on AI, activity and outcomes brings together research on writing versus shipping code, what AI lets developers attempt, code review and autonomy. That framing helps separate questions that a single productivity claim can blur. Producing code, exploring a possibility and delivering a dependable change call for different evidence.

Start the ticket with the decision the answer will inform. Can this integration preserve the access rules we need? Does this workflow remove a real source of confusion? Who will choose whether to proceed? If nobody can name the decision, sending an agent to build a demonstration is premature. There is no clear result to judge.

A useful answer connects observed evidence to the question and states what remains unknown. Keep the starting conditions, the attempted approach, the result and the limitations together. Have someone other than the builder judge whether that evidence supports the conclusion. A persuasive explanation from the agent is still a claim to check.

Imagine an experiment asking whether a replacement search service can enforce your account permissions. The agent builds a small isolated test and finds that forbidden results still appear. Preserve the requests, access rules and observed responses. A decision to reject that approach can complete the experiment even though the implementation belongs nowhere near production.

That result has a boundary. It rules out the approach under the conditions tested; it does not establish that every search service will fail or that the product cannot support better search. Record the limit beside the decision so the next agent inherits evidence it can use without inheriting a sweeping conclusion nobody proved.

Fix the experiment's question and stopping condition before work starts, and keep its cost visible when the answer disappoints you. Name the allowed data, systems, spend and time. If the limit arrives before the evidence can settle the question, record the result as inconclusive. Do not award success for reaching the budget.

Keep delivery failures in the delivery record. If a task promised a working search feature and failed its access checks, it missed that promise. The failure may teach you something valuable, but it does not become a successful experiment after the fact. Authorize any follow-up investigation separately and retain the original rejection.

This extends the distinction in AI usage and factory performance: activity helps explain an outcome but cannot substitute for it. Keep accepted experiments, inconclusive experiments and failed delivery visible as distinct results. Combining them into a cheerful completed-work total removes the information you need to decide what the factory should attempt next.

Retain the evidence and decision, then either retire the prototype or authorize a separate production change. A successful investigation earns its conclusion. It does not automatically earn permanent credentials, a supported endpoint or a place in the product. Those commitments need an owner and their own acceptance conditions.

If the answer justifies building, create a delivery ticket that names the behavior, existing obligations, independent checks and release decision. Reuse code only if it passes those requirements. Disposable code needs a hard boundary because shortcuts that were safe inside an isolated trial can become maintenance costs when copied into a live system.

If the decision is to stop, remove temporary resources and revoke trial access. Preserve enough evidence to explain why the approach was rejected and when reconsidering it would make sense. The factory should not keep an abandoned experiment running just because deleting it would make the output chart look smaller.

Leaders should track decisions resolved by experiments separately from changes accepted for delivery. For an experiment, record whether its question was answered, what decision followed and what the attempt cost. For delivery, retain the promised behavior, acceptance evidence and production result. Both records should include unsuccessful attempts and human intervention.

Do not invent a monetary saving for every rejected idea. You can show that evidence informed a decision to stop without claiming to know what an unbuilt feature would have cost. Cheaper code makes selection more important, and a defensible refusal is useful even when its value cannot honestly be expressed as a return percentage.

Gunner Technology's recommendation is to give exploration a clear contract at intake. People decide which uncertainty matters and whether the answer is convincing. Agents build the bounded trial, gather the evidence and clean up. Keep that route inside the accounts you own, and let a well-supported decision to stop count as the result it actually is.