2026-11-17Gunner Technology
A Short Prompt Can Still Run Up the Bill
A shorter prompt can make an individual request cheaper, but it does not guarantee a cheaper coding-agent run. Compare the full sequence of requests and the work that passes before calling the change a saving. A tidy opening prompt can still lead to an expensive afternoon.
What did the harness benchmark find?
In Runta’s FrontierHarness evaluation, steps tracked cost per pass more closely than initial prompt size or tool count did. That association appeared across the harnesses tested. A harness is the software that supplies a model with instructions, executes its tools and manages its working loop. The finding concerns the whole loop, not just its opening request.
The article defines a step as an entry in the recorded sequence of agent actions. It calculates cost per pass from average task cost divided by pass rate. That matters: the comparison asks what it costs to obtain passing work, rather than rewarding a small prompt or a cheap attempt that leaves the task unfinished.
The limits matter. Runta used open models through a shared inference provider and estimated costs at list prices. The association does not prove that removing an arbitrary step improves efficiency. Nor does it establish that a smaller prompt never helps. It gives you a reason to inspect repeated activity before treating prompt length as your main cost control.
Why can a small prompt still lead to a large bill?
A small opening prompt can still lead to a large bill because later requests carry the context accumulated during the work. Context means the instructions, task material and earlier exchanges available to the model. Runta points to repeated context processing as the mechanism connecting more steps with more usage. The opening request captures only the start of that sequence.
Consider a hypothetical harness that offers a shell but omits the repository’s usual validation command. The agent searches for commands, guesses one, repairs the failed invocation and searches again. A slightly longer instruction that names the correct command might avoid those turns. That is an example to test, not a measured result from the article.
The reverse is possible too: irrelevant instructions can add input without helping the agent finish. Keep the context that changes a decision and remove material that does not. Our guide to budgeting repository exploration explains how to ask whether a search produced a useful finding. Apply that question to the sequence, not merely its length.
Which agent turns should you try to remove?
Try removing turns that repeat work without adding a finding, a change or required evidence. Look for the same failed tool call with unchanged inputs, a broad search repeated after the relevant file was found, or a repair cycle that never addresses the reported defect. Each is a candidate for investigation, not an automatic deletion rule.
Do not classify verification as waste because it happens after coding. Running the required checks, investigating a failure and confirming a repair can all be necessary work. A shorter run that stops before the verdict is incomplete. Keep the acceptance rules outside the builder’s authority, so a cost target cannot quietly become permission to skip proof.
Fix the cause of a repeated turn. Return a usable error when a command fails. Supply the missing entry point when navigation is the problem. Ask the product owner when intended behavior is unresolved. Gunner Technology recommends changing the machinery that produces the repetition before asking the agent to hurry. Otherwise, you may just get an earlier claim of completion.
How do you test whether the change saves money?
Test the change against the unchanged harness on the same representative tasks, with the same model, provider, permissions and acceptance checks. Change the suspected cause separately from unrelated settings. Repeat the comparison and retain the request records, total usage, elapsed time and independent verdict for every attempt, including failures.
Compare the entire route. Did the candidate finish with fewer requests, or did it merely push investigation into a retry or another worker? Did smaller tool results omit evidence the agent later needed? Keep those consequences attached to the task. Keep the cheaper version only when the agreed work still passes.
Record cached input separately from other input, because repeated context does not always carry the same price. Your cache belongs in the comparison, including the conditions under which reuse actually occurred. A reduction in request count is evidence about the route; measured cost and acceptance tell you whether that reduction helped.
What should you ask before buying a leaner harness?
Ask for complete task records showing where the advertised saving came from. A shorter system prompt, fewer tools or a smaller codebase describes the design. It does not tell you whether the agent reaches the same accepted result with less spending. Ask which repeated actions disappeared and which checks remained.
Runta’s result makes that buying question sharper without choosing your production setup for you. Start with a bounded workload in a repository you own. Let the factory capture usage and expose repetition. People decide which outcomes are worth funding and whether the returned work is right. Approve the change when that record supports it.
In response to In 27,280 coding agent runs, 64% of inference spend went to runs that failed by Runta.
Quick answers
Should we remove a tool that the agent rarely uses?
Only after checking which tasks need it and testing the replacement route. Rare use can still protect an important recovery or verification path.
Should we set a maximum number of agent steps?
A step limit can bound an attempt, but reaching it must preserve an incomplete outcome and the evidence collected so far. It cannot turn unfinished work into a pass.
Does a high cache hit rate prove that the harness is efficient?
No. Cached input can reduce processing cost while the agent continues repeating unproductive work. Judge the complete cost of the accepted result.