2026-11-17Gunner Technology
Make the Shortcut Prove It Knows Where to Stop
Test a rule-based AI fast path against labeled requests, including cases it must leave for a model or a person. Then run it without acting on its decisions and check the results before giving it live traffic. Skipping a model call saves nothing if the shortcut sends work to the wrong place.
What does a rule-based fast path actually do?
A rule-based fast path handles requests that explicit code can settle before the workflow calls a model. Deterministic code follows the same written rule for the same input. That makes its behavior repeatable; whether the rule captures the intended decision is a separate question.
The AWS article on reducing generative AI costs describes checking rules first, using constrained classification for fixed choices and reserving generation for open-ended work. It also describes having a model draft a reusable rule that people review before deployment. These are different ways to avoid asking a large language model to do every step.
Consider a hypothetical intake service that recognizes a documented request code. An exact match can select a queue. Free-form prose about several problems may need interpretation. The useful boundary is the point where the available facts stop settling the route. Define that boundary before measuring how many model calls disappear.
Which cases must the shortcut pass on?
The shortcut must pass on any case its rule does not fully settle. Write those cases into the test set alongside successful matches. A test set is a collection of inputs with expected results; here, some expected results should explicitly say that the rule cannot decide.
For the hypothetical intake service, include an unknown request code, a missing field and conflicting codes. If you are matching words instead, test negation, quoted messages and requests with several intents. A message that says it does not need a refund still contains the refund keyword. These examples test the meaning you assigned to a match, rather than whether the matching function runs.
Have the person who owns the workflow confirm the expected outcomes. Keep examples the rule writer did not use while developing it, and test those before promotion. Our guide to enforcing agent instructions in code covers making a rule unavoidable. This check asks whether that rule makes the right decision in the first place.
How do you compare routes without duplicating actions?
Compare proposed decisions while allowing only the existing approved route to act. This is shadow testing: the candidate observes a permitted copy of the input and records what it would do, without sending a message, updating a record or triggering another workflow.
Record the input category, rule version, proposed route and actual outcome. Inspect disagreements against the approved requirements and labeled examples. Agreement with a model is useful evidence, but the model can also be wrong. A disagreement should create an investigation, rather than automatically replacing the rule with whichever answer sounds more convincing.
Keep classification separate from permission to act. Recognizing that a request concerns a refund does not authorize a payment. The receiving workflow still needs its authorization checks. As Confidence Needs a Cost Function explains, a routing decision needs limits based on what a mistake could cause. The same discipline applies when code makes that decision.
What proves the fast path is cheaper?
The fast path is cheaper when it reduces the measured cost of correctly completed work under the same requirements. Count execution, fallback model calls, corrections and human intervention within a stated measurement boundary. Track response time through completion, too. A quick initial route can still leave a request waiting in the wrong queue.
Keep missed opportunities separate from wrong decisions. A request unnecessarily sent to a model costs extra computation. A request incorrectly settled by code may produce an incorrect action or require recovery. Combining both into a single error rate makes the more expensive failure harder to see. Report the outcomes by route and consequence.
Use complete batches, including failed and unfinished requests. Our explanation of why median cost per pass cannot predict a budget shows why a typical successful run cannot stand in for total spend. Apply that lesson here: fewer model calls are an intermediate result. The operating result is less spending for work that still meets the requirements.
When should you narrow or disable the fast path?
Narrow or disable the fast path when corrections reveal that its assumptions no longer hold. New input formats, changed product policy and unfamiliar request categories should trigger another evaluation. Keep the rule version attached to each decision so an owner can identify the affected work.
Start with a bounded category and retain the previous approved route. Before launch, name who can turn the shortcut off and what evidence should prompt that decision. Continue sampling its decisions after release, including apparently successful matches. A rule that never reports uncertainty can still be confidently sending work to the wrong place.
Gunner Technology recommends starting with the decisions you can specify and test clearly. People define what correct handling means and judge unresolved cases; the factory runs the checks, records the decisions and routes work within those limits. Let the shortcut handle the cases it has proved it can settle, and make the rest visible.
In response to Cut generative AI costs: Code for rules, models for judgment by Artificial Intelligence.
Quick answers
Can an AI agent write the routing rule?
Yes, but review the rule and verify it against independently labeled cases before deployment. Generating code does not establish that its decisions match your policy.
Should every request that misses the rule go to a larger model?
No. Choose the approved next step for that kind of uncertainty, which may be a classifier, clarification or human judgment. Missing information may require a question rather than more computation.
Do we need to keep the model running beside the rule forever?
No. After evaluation, use a defined sampling plan and repeat the comparison when inputs or requirements change. Keep enough evidence to detect incorrect decisions that appear successful.