Your Harness Upgrade Needs a Cache Check

An upgrade can preserve the answer while changing how much earlier input the model has to process again. The harness is the software that builds requests, runs tools and carries the conversation forward. Prompt caching lets a provider reuse processing for matching earlier input. The prefix is the beginning of that input, where shared instructions and tool definitions often sit.

In its analysis of harness efficiency, Neuralwatt traced reproducible cache breaks in its lab to edits of previously submitted material. It also reported production misses it could not fully attribute, including possible cache eviction. That distinction matters: a miss deserves investigation, but it does not automatically prove your harness changed the prompt.

Gunner Technology's position is that request construction deserves a regression check: a repeatable test that catches behavior an upgrade makes worse. You already ask whether the agent still finishes the task. Ask whether an ordinary transition now makes it pay to process a long history again. Those are different failures, and a passing task test can miss the second.

Replay the transitions your agents actually encounter after the conversation has grown. Build a synthetic transcript with tool calls and results, then exercise resume, a settings change, a tool change and compaction. Compaction means replacing part of the history with a shorter summary. Include it if your production route uses it. Keep confidential conversations out of the fixture.

Separate request construction from live execution. Feed the same recorded inputs into each harness version and compare the requests it builds. That isolates changes introduced by the harness without letting different model answers rewrite the experiment. Record the earliest changed message or tool definition and whether the change was expected. When provider-side rendering is hidden, acknowledge that limit.

Then run a bounded live replay to check the provider's reported usage. Hold the model, task, tool results and relevant settings steady, and record differences you cannot control. Warm each candidate's cache deliberately. Run the transition late enough to expose the cost of disturbing accumulated history. A short greeting followed by a short reply won't test that failure.

Measure cache reads, cache writes, uncached input and response delay around each transition, alongside the task result. Keep the provider's definitions attached to the record. Anthropic documents separate usage fields for input read from cache, input written to cache and input outside those categories. Don't combine unlike counters into a reassuring hit rate.

Compare the affected turn and the work that follows it. A healthy average across the session can conceal an expensive break near the end. Match a changed request boundary to a rise in newly processed input when the evidence supports it. If the request stayed stable, investigate cache availability and provider conditions before blaming the upgrade.

Keep energy claims separate from token accounting. Neuralwatt's report includes measured energy; your usage counter may not. Report what your system actually measures. Our earlier model-cost comparison covers choosing a route under different cache conditions. This check asks whether changing the machinery quietly damaged a route you already chose.

Stop the rollout when a representative transition exceeds its agreed cost or delay allowance without an accepted explanation. Set that allowance from repeated baseline runs and the workload's requirements. Keep the baseline and acceptance rule outside the upgrading agent's editable files. Otherwise it can make its own regression disappear by changing what passes.

Some invalidation is necessary. A new security instruction must reach the next request even if reuse suffers. If a tool loses permission, revoke access in the tool execution layer immediately; preserving its old definition must never preserve its authority. Correctness and access controls decide what may stay stable. The cache does not get that vote.

An intentional increase needs a named reason, evidence that the changed behavior works and approval from the person responsible for the tradeoff. An accidental rewrite needs a fix and a replay. Turn the discovery into a lasting check, using the learning loop to make the next upgrade inherit the lesson.

Start with a recurring job in a repository you own. Save a safe replay fixture, the harness versions, request comparisons and provider usage beside the upgrade evidence. Keep the existing functional and permission checks in place. Lower input processing is useful only when the agent still follows the current rules and produces acceptable work.

After approval, send a limited share of that queue through the new harness and watch the same transitions in production. Keep a route back to the previous version. Lab evidence can establish that a request builder stopped rewriting history; production observation tells you whether the expected benefit survives actual session lengths and cache conditions.

People choose whether the upgrade earns its place. The factory should collect the evidence and stop an unexplained regression before it spreads. A harness update changes the machine every agent works inside. Give that change a check worthy of its reach.