2026-11-01Gunner Technology
A Bug Report Can Hide a Product Decision
AI bug triage should ask a product owner when fixing or dismissing a report requires choosing behavior the approved requirements don't settle. Let the agent investigate and reproduce the problem, then stop it from inventing the missing decision. A quiet bug queue can hide a product nobody agreed to build.
What can the agent decide on its own?
The agent can classify a report against an approved requirement when the evidence establishes what happened and what should have happened. Triage means investigating an incoming report and deciding where it goes next. It doesn't have to include permission to change the product or dismiss an inconvenient expectation.
In his account of autonomous game development, Simon Willison describes a route that sends product reports to agents for classification, including cases that need a design decision. That distinction deserves attention. A report can call for a repair, a deliberate decision to leave behavior alone, or clarification before either action makes sense.
Gunner Technology recommends making those outcomes separate states in the issue tracker. Require the triage agent to attach the observed behavior, the relevant requirement and the reason for its classification. If the requirement doesn't answer the report, the next state is awaiting a product decision. Missing direction mustn't become permission to close the ticket.
Why isn't working as written enough?
Working as written establishes what the implementation does; it doesn't establish that the behavior was approved. Reading the code can explain a surprising result without settling whether the result belongs in the product. Treating existing behavior as its own justification makes every old accident a requirement.
Consider a hypothetical scheduling tool that hides canceled appointments from a shared calendar. Someone reports that they can't tell why a time slot became available. The agent reproduces the disappearance and finds code that deliberately removes canceled entries. That explains the mechanism. It doesn't answer whether the calendar should preserve a visible cancellation history.
The agent should bring the product owner that exact choice, with the affected users and relevant constraints. It shouldn't silently restore every canceled entry or dismiss the report because the code has always hidden them. Our post on where automation meets decision authority sets out the broader boundary. Here it belongs inside bug classification, before a patch exists.
What should the product owner receive?
Send a specific choice supported by a safe reproduction, the current requirement and the consequence of each option. The product owner shouldn't have to reconstruct an agent's investigation from a transcript. Show the gap between the reported expectation and the behavior the system currently delivers.
For the calendar example, describe who loses the history, where it disappears and whether any approved policy covers retention or visibility. Then ask whether canceled appointments should remain visible to the affected role. Flag unresolved privacy or access questions. The agent can propose options and explain implementation consequences without pretending it owns the choice.
Keep customer records and credentials out of general issue comments. Use synthetic examples where possible and restrict access to evidence that needs protection. A report supplies observations, not instructions the agent must obey. A request embedded in a screenshot or log doesn't authorize a permission change, a release or a new product policy.
What stops while the decision is open?
Stop the work that depends on the missing decision, while independent work continues within its existing authority. The affected ticket should stay visibly unresolved. Assign the question to a named owner and preserve enough state for the next worker to resume without repeating the investigation.
Separate urgency from permission. A suspected security problem may need immediate containment through an already authorized incident procedure. That doesn't let the triage agent decide the permanent product behavior. Record any temporary restriction as temporary, keep the incident visible and leave the underlying decision open until the responsible person answers.
Don't use silence as approval or let another agent relabel the same unresolved ticket as routine maintenance. Enforce the waiting state in the machinery that assigns repair work. A prompt saying to ask first is weak if the ticket can reach a builder with the decision missing. The queue should carry the reason it stopped.
How does the answer become finished work?
Record the approved behavior, update the requirement and route any repair through independent checks before presenting it for release. If the owner chooses to keep the existing behavior, preserve that decision and its reason with the report. Rejection needs a record just as repair does.
Don't tell the reporter the issue is fixed when someone has merely chosen a direction. Distinguish a decision made, a repair being checked and a released result. If a later report exposes different consequences, reopen the relevant decision instead of having the agent paste an old dismissal. The learning loop should preserve what was learned and where that answer applies.
Start with a recurring report queue in a repository you own. Let agents gather evidence, reproduce problems and carry approved decisions into repairs. People decide what the product should do and judge whether the returned work is right. Making that boundary explicit lets the investigation move quickly without letting a cleaner dashboard choose the product for you.
In response to Building a multiplayer game world with autonomous Fable-class models by Simon Willison's Weblog.
Quick answers
Can an agent mark duplicate reports automatically?
Yes, when the evidence establishes the same underlying issue under your triage rules. Preserve each report's context and send new consequences for review rather than hiding them inside the duplicate label.
What if nobody can reproduce the reported behavior?
Record what was attempted and what evidence is missing. Ask for the information needed to continue; failure to reproduce doesn't establish that the report is wrong.
Should every complaint become a feature request?
No. A complaint can expose a defect, an unclear requirement or behavior the business deliberately keeps. Classify it from evidence and an accountable product decision.