Model Routing in Practice

Updated 30 June 2026

Added Cognition's Devin Fusion (announced 29 June, the day this published) to the routing landscape and the close: a frontier-plus-sidekick router Cognition says now drives 88% of its own merged pull requests.

A few weeks ago I sat and watched Claude Opus 4.8 think. It had decided that renaming one helper function and fixing its three call sites deserved a short paragraph of deliberation first, weighing whether the new name might collide with something elsewhere in the module. It would not. The whole job was a thirty-second edit. Opus took the scenic route, and I let it, because Opus was already what my prompt was pointed at and switching felt like more bother than the task deserved.

I am on the $100 Claude Max plan ^{[R1] [R2]}, so the meter does not tick where I can see it. That is the trap. The waste did not arrive as money. It arrived as latency, and as a low background sense that I was using a model built to reason about distributed systems to do the rough equivalent of alphabetising a drawer.

I had made the argument myself, not long before, that for most of a working day you no longer need the frontier model at all.

Reading my own piece back, the gap between what I believed and what I did was hard to miss. I believed in routing. I was not routing. I had one default and a vague intention to do better, which is not the same thing.

Fit comes before price

When people talk about using cheaper models, the framing is almost always cost: swap the expensive one for the cheap one wherever you can get away with it. That framing is real, and it buries the more useful question underneath it.

The models on my shortlist are not strung along a single line from worse to better. They have different shapes. One is good at holding a plan in its head and steering a long multi-step change without losing the thread. One is fast and precise when the task is pinned down to the last detail, and goes vague and confident the moment it is not. One can read a screenshot. One holds a million tokens of context. The same model is often excellent at one of these and clumsy at another. Choosing by price alone is choosing on one axis of a problem that has five or six.

There is one more axis, and it is the one that reshaped how I think about the cheap models. Andrej Karpathy put the principle simply: these models can automate what you can verify ^[R19]. Some tasks carry their own check. Code compiles or it does not; an extraction is valid JSON or it throws. He leaves the next step unsaid; I will take it. When the check is mechanical, you can hand the work to a small, cheap, even local model and let it fail cheaply, because a compiler or a quick retry catches the miss in seconds. When the check is not mechanical, a wrong-but-fluent answer to a creative or multi-hop problem sails straight past you, and that is where the frontier model earns its price. Verifiability, not raw difficulty, is often the cleanest line between the cheap tier and the dear one.

So the question each time is not which model is cheapest and still works. It is what shape the task has, how easily I can check the result, and which model fits both. Cost comes along as one input, often a decisive one, but not the first thing I reach for.

The 2026 bench

What makes this worth the bother in mid-2026 is that the spread of capability per dollar has become enormous. A year ago the open-weight models were a noticeable step behind and the price difference was a rounding error. Now the price difference is a chasm and the capability gap, on the right kind of task, has mostly closed.

Here is the part of my shortlist I reach for most, with what each one is genuinely for and what it costs to run.

_{Prices and model versions current at June 2026. The cast turns over fast; treat the numbers as a snapshot, not a constant.}

The headline figure is the gap. Opus 4.8 lists at $25 per million output tokens ^[R1]. DeepSeek's V4-Flash lists at $0.28 ^[R3]. That gap is a different order of magnitude: close to ninety times cheaper per token of output for the kind of bounded work it handles well. Even DeepSeek's larger V4-Pro, at $0.87, sits around thirty times under Opus ^[R3]. The much-repeated "open weights at a tenth of the cost" line turns out to be conservative.

The shift reaches inside the tools too. Microsoft shipped MAI-Code-1-Flash at Build 2026, a 5B model that now appears as one of the default models inside VS Code, sized in the same class as Haiku and aimed squarely at the quick-edit tier ^[R10]. A 5B model won't match the frontier on reasoning. It answers instantly and costs almost nothing, and a large fraction of editing is exactly that: small, local, well-defined.

Read the benchmark before you trust it

Most of the open-weight scores you see quoted are vendor-reported on SWE-bench Verified, the easier of the two standard suites, where the strong open models land around 78 to 81% ^{[R3] [R7]}. On the harder, independently-graded SWE-bench Pro the picture is soberer: the open entries sit roughly between 38 and 59%, behind the proprietary leaders at around 52 to 59% ^{[R10] [R11]}. DeepSeek and MiMo have independent backing for their numbers; GLM, MiniMax and the hosted Qwen flagship are mostly graded by their own makers ^[R11]. The cost story is real. The parity story needs the asterisk.

What I actually run

My daily driver is still Opus 4.8, and I am not going to pretend otherwise. The fixed plan makes it the path of least resistance, and for the planning and review work that fills a lot of my day it earns the slot. For code review I reach, sparingly, for Codex, which runs on GPT-5.5 and reasons across the change rather than just linting it ^[R12]. That pairing is deliberate, and it is not about one being better. Codex has a different temperament. Handing my own work to a model that thinks differently catches things that a second pass from the same model never would.

Underneath that I keep opencode wired up to a row of open-weight models, mostly so I can find out what they are actually good for rather than what their launch posts claim ^[R17]. GLM 5.2 has been the pleasant surprise: a genuinely capable all-rounder I trust on real work ^[R4]. Qwen's coding line is excellent when the task is specified down to the corners, and noticeably worse when it has to infer what I meant ^[R18]. DeepSeek V4-Flash is so cheap and so fast on mechanical work that the only reason not to use it is forgetting it is there.

And then the honest part. MiniMax and MiMo, I struggle to steer. The capability is clearly in there, but my prompts that work fine on Opus produce mixed, wandering output from them, and I have not yet put in the hours to learn their grain. That is the lesson hiding in the whole exercise: a cheap model is only cheap if you can steer it. The token price is the visible cost. The hours you spend learning to drive a model well are the invisible one, and they do not show up in anyone's pricing table. I have a little private inference set up for Gemma 4 and gpt-oss as well ^{[R8] [R9]}, and I will admit I have barely touched it, which tells its own story about how much friction a route has to remove before I will actually take it.

A policy that fits the work

Out of all that, a rough policy fell into place. Not a config file, more a set of reflexes.

Mechanical, repetitive edits go to the smallest fast model that can do them. Bounded, well-specified coding goes to a specialist that is strong on exactly that and cheap to run. Planning, architecture, and review stay with the frontier model, because that is where its extra reasoning earns the money and the wait. Anything that needs to see an image or hold a very long context gets routed by that need, not by habit, since most of my models cannot do either well.

Locality is its own gate, sitting across all of that. Some work simply should not leave the building, regardless of which model would do it best. That decision comes first and overrides the rest.

The unit that matters here is cost per task, not cost per token. A reasoning model that burns ten times the tokens to get a hard thing right in one pass can be cheaper than a cut-rate model you have to correct three times. The per-token price on the table above is the input to that sum, not the answer. And the precondition for any of it is that the work is broken down far enough to have a shape at all. You cannot route a task you have not decomposed. A vague "build me the feature" has no shape, so it defaults to your most capable model by reflex, which is precisely how I ended up watching Opus rename a function.

How people are actually routing

Go looking and you find the routing itself has become a small industry. It splits into a few families worth telling apart. The simplest are embedding routers: Aurelio's open-source Semantic Router compares your request against a set of example phrases in vector space and picks a lane with no extra model call, which it clocks at about a tenth of a second against roughly five seconds to ask an LLM the same question ^[R21]. Then there are routers that classify the prompt before committing, with a small LLM judge or a lightweight difficulty classifier estimating how hard the task is and escalating to the frontier only when it looks worth it. A cascade does the same thing the other way round: run the cheap model first, escalate only when the answer fails a check or comes back unsure, which is the verifiability rule from earlier turned into control flow. At the far end is fusion, which gives up on choosing at all: OpenRouter's Fusion fans a prompt out to a panel of models and has a judge synthesise one answer, and reports the fused result beating the best single model, though that is the vendor's own benchmark and reads best as such ^[R22]. Sakana's Fugu pushes the idea further, packaging a coordinated set of agents behind one endpoint ^[R23]. Cognition's Devin Fusion, announced the day this piece went up, wears the same word for a different design: instead of fanning one prompt across a panel, it runs a frontier agent and a cheaper sidekick side by side, each with its own cached context, and switches between them mid-task at the compaction boundary so the swap does not blow the cache ^[R27]. Cognition reports it holding frontier-level quality at roughly 35% lower cost on FrontierCode, its own new benchmark. Even that scoreboard, though, shows the cheaper fused run a few points under a pure frontier pass, so the lower cost comes with a small quality give-back ^[R27]. Beneath all of it sit the gateways and proxies: OpenRouter as a hosted socket onto hundreds of models ^[R15], and LiteLLM as the open-source proxy teams self-host to load-balance, fail over, and cap spend across providers ^[R26].

I would not oversell any of it. A large routing benchmark early this year found that several of these methods, the commercial OpenRouter router included, did not beat just sending everything to the best single model ^[R24]. That result is already months old, which in this field is most of a lifetime, so I hold it loosely. But it lands on something durable: routing is a real engineering problem, not a free win. When OpenAI wired an automatic router into GPT-5 the year before, it sent enough work down the slow path that users revolted and the company walked parts of it back ^[R25]. A router that picks wrong is worse than no router, because the failure is now invisible too.

A quiet problem for the frontier

Every task you route away from the frontier model is a task its maker no longer bills you for, and the companies selling the expensive models know it. By one account around 95% of enterprise AI spend still flows through the priciest frontier models, even on work a cheaper one would handle, while the coding-agent vendors claim five to ten times better cost efficiency on routine jobs once you stop ^[R20]. If routing catches on at that scale, the frontier labs keep the genuinely hard problems and lose the long tail of ordinary ones. That is a thinner business than the one their valuations assume.

I should be honest, though, that routing by merit is partly a luxury, and that is what protects the labs in practice. Plenty of the organisations I work with do not get to pick the best model for each task on a Tuesday afternoon. They sit inside a Microsoft or a Google estate, and the model is chosen for them by governance, data residency, and the billing relationship already in place. If your tenant runs on Azure and your data cannot leave it, the Microsoft model is the model, and whether something benchmarked half a point higher this month is not the deciding question. The routing policy above is a privilege of people whose hands are not already tied. The frontier's quiet good fortune is how many hands are.

Route the work, not the model

The thing I keep half-forgetting is that the careful hand-routing I have been describing is already on its way out. Claude Code will plan on Opus and switch itself to Sonnet to implement, on one command ^[R13]. GitHub Copilot has an automatic mode that picks the model for you ^[R14]. Cognition, whose CEO I cited earlier claiming five-to-ten-times better cost efficiency on routine work, now reports that 88% of its own merged pull requests are driven entirely by Devin Fusion's automatic router ^{[R20] [R27]}. The routers from a moment ago are being pulled inside the tools, and one layer up, meta-harnesses now sit above several agents and route between them as interchangeable parts.

So the skill I have been sharpening is, in part, a transitional one. The model picker is being absorbed into the harness, the way query planning got absorbed into the database. What will not be absorbed is the work underneath it. A router can only route a task that has been broken down into something with a recognisable shape, and that breaking-down is mine to do. The better I get at decomposing a job, the less the model choice feels like a decision at all, because each piece arrives already wearing a label that says which line it belongs on.

Which is the part I had backwards that afternoon, watching the most expensive model I own deliberate over a rename. I thought the fix was to get better at choosing models. The fix was to get better at cutting the work into pieces small and clear enough that the choice mostly makes itself. Route the work properly, and the model follows.