Use the Pareto Frontier to Shrink the Trial List

The Pareto frontier contains candidates where improving a measured result requires giving something else up. A candidate falls off the frontier when another is at least as good on every chosen measure and better on at least one. That relationship is called dominance. It lets you discard an option without deciding how much extra quality is worth in extra spending.

The Artificial Analysis page for Kimi K3.1 Pro separates intelligence, cost per task and speed, and includes a Pareto-line comparison. Those are different questions. A model can look attractive against cost and less attractive against completion time. The axes define the trade-off you're looking at.

Gunner Technology's position is that the frontier belongs at the entrance to your evaluation process. Use it to reduce the field. Keep the chart's date, evaluation method and measured conditions with the shortlist, so someone can explain later why a candidate made it in. Don't turn a temporary comparison into permanent model loyalty.

Compare models that can meet the job's hard constraints before weighing their trade-offs. A hard constraint is a condition you won't trade away: permitted data handling, required tools, available access or a completion deadline. A model outside those limits isn't a bargain for that job, regardless of where it sits on a public chart.

Suppose you're choosing a worker for a queue of routine repository updates. The job needs tool access, a defined spending allowance and finished work before the review window. Exclude routes that can't satisfy those conditions. Then compare the eligible candidates. A public frontier may omit an option that becomes useful after your own constraints remove its apparent superior.

Write down what qualifies as a separate candidate. A model reached through a different provider or run with different tools is a different operating route. Don't combine its best observed cost with another route's best observed speed. That would create an option you haven't actually tested and might not be able to buy.

Keep a candidate when the apparent disadvantage could be measurement noise or when an important property hasn't been measured. A tiny difference in a trial doesn't establish durable dominance. Repeat representative jobs and inspect the spread of results before claiming that a route has no remaining advantage.

Look at what disappears when you average. A model that struggles with unfamiliar code might still suit repetitive edits in a well-understood repository. A faster average can also conceal runs that miss the deadline. Keep task classes separate when their consequences differ, and retain failures in the comparison rather than counting only successful runs.

Our post One Score Cannot Route a Factory explains why an aggregate ranking can't assign every job. The narrower point here is about exclusion: don't remove a candidate from all consideration because it loses on a mixture of tasks you don't intend to give it.

Measure how often each route delivers an accepted result, what the whole route costs and how long it takes. Acceptance means the result passes checks set outside the builder's control. Give candidates the same repository snapshot, task instructions and authority, and record any adaptations required to make their tools work.

Keep rejected attempts and rescue work in the bill. A route that needs another worker to finish has consumed both workers. Record unfinished jobs as failures to deliver within the limit; don't quietly drop them from the average completion time. Otherwise the option that abandons difficult work can appear to be your fastest worker.

Context reuse belongs in those conditions too. As Your Cache Is Part of the Model Comparison explains, a saving that depends on repeated input may disappear on a cold start. Compare the operating conditions your queue will actually produce. Then plot the trade-offs that remain after the results pass the required checks.

A person chooses the trade-off the business will accept, and the factory applies that choice to the defined workload. The frontier deliberately leaves that judgment unresolved. If a cheaper route takes longer and both meet the deadline, decide whether the saving matters. If time matters more, say so before the next ticket arrives.

Start with a bounded queue in a repository you own. Keep the current route available while the candidate earns its place. Preserve the evidence for excluding other options, including the task class and conditions involved. Losing the primary job doesn't automatically make a model useless as a fallback when a provider is unavailable.

Reopen the shortlist when a relevant condition changes: the workload, provider access, price, deadline or acceptance results. The factory should flag that change instead of making someone remember an old comparison. The useful output of the exercise is a smaller set of defensible choices and a recorded decision about which trade-off you're buying.