Stop Using Your Biggest Model for Everything

The safest career move on any AI project is to default to the biggest model for everything. Nobody gets blamed for choosing the most capable option. The bill arrives later, once usage has scaled past the point where anyone remembers the decision that caused it. Two of 2026’s most-discussed corporate AI stories are exactly this pattern, playing out at real scale, months apart.

Two companies found out what “no tiering policy” costs

Uber’s CTO, Praveen Neppalli Naga, told The Information that surging use of AI coding tools had maxed out the company’s full-year AI budget just months into 2026. Claude Code adoption inside Uber’s roughly 5,000-engineer organization jumped from 32% to 84% between February and March alone — the company had been ranking engineers on internal leaderboards by usage, which is exactly the kind of incentive that rewards reaching for the biggest tool regardless of the task. By the time anyone looked closely, about 11% of real-time backend code updates were AI-generated, up from under 1% three months earlier. Naga’s own words: back to the drawing board.

Microsoft hit a version of the same wall from the other side. It rolled Claude Code out to roughly 5,000 engineers across its Experiences + Devices division — Windows, Microsoft 365, Outlook, Teams, Surface — in December 2025. Adoption climbed to 84–95% within four months. Per-engineer API costs ran $500 to $2,000 a month, well past what flat-rate budget planning had assumed. EVP Rajesh Jha’s internal memo ordered a switch to GitHub Copilot CLI by June 30. Neither company’s problem was that the tool didn’t work. Both problems were the same: every task, regardless of what it actually required, ran through the most capable and most expensive option available, because nothing in the system said otherwise.

Most of what a system does isn’t the hard part

A real AI workload is rarely uniformly difficult. It’s a mix — some genuine multi-step reasoning, surrounded by a much larger volume of classification, extraction, formatting, and routine drafting that doesn’t need frontier capability to get right. Treating the whole mix as if it were the hard 10% is where the waste comes from.

Put faces on that mix and the split stops being abstract. Renaming a variable, reformatting a log line, pulling three fields out of a support ticket, drafting a first-pass email — none of that benefits from frontier reasoning. It benefits from being fast and cheap, because there’s a lot of it and any single instance is low-stakes on its own. Untangling a race condition across three services, deciding how a schema migration should sequence against live traffic, drafting the actual compliance language a customer will read — that’s the other 10%, and it’s exactly where the reasoning tier earns its price. The mistake was never using the big model. It was using it for the first list because it’s also correct for the second one, and nothing in the system forced the distinction.

Chen, Zaharia, and Zou’s FrugalGPT, out of Stanford, built this into an actual system: an LLM cascade that routes each query to the smallest model likely to answer it well, escalating to something larger only when the cheaper model’s confidence is low. Their result: matching GPT-4’s accuracy at up to 98% lower cost, or beating GPT-4’s accuracy by 4% at the same cost. LMSYS’s RouteLLM operationalized the same idea as an open-source router, trained on real preference data, that sends a query to a cheaper model when it’s confident the answer will hold up and to a stronger one when it isn’t. Neither project is exotic research. Both exist because the price gap between tiers is real enough to be worth automating around — OpenAI’s GPT-4o costs roughly 16.7 times more per token, on both input and output, than GPT-4o-mini. The same workload can cost $45 or $2.70 depending on which one it ran through.

Tiering has two axes, not one

Cost and complexity aren’t the whole story. The other axis is stakes: what happens if this specific answer is wrong. IBM’s own framing of this is direct — risk should determine the architecture, not the other way around, with routine, high-volume work going to smaller or open-weight models and high-impact decisions routed to the most reliable available option, paired with human review, deterministic rules, or independent verification. A more concrete version of the same rule, aimed at regulated industries: route high-volume, low-risk work — summarization, intake classification, internal drafting — to smaller models, and keep the reliability premium, plus human review, for anything touching underwriting judgment, compliance communication, or a customer-facing decision. A task can be simple and still deserve the expensive tier, if getting it wrong is expensive. A task can be moderately hard and still not deserve it, if getting it wrong is cheap to catch and fix.

Routing down has a cost too — and it’s the one nobody budgets for

Tiering isn’t free money. It trades a cost you can see — the bill — for a risk you mostly can’t, until it’s already compounded. RouteLLM’s own paper is honest about exactly where that trade breaks: on structured extraction, classification, and summarization, quality loss from routing to a weaker model is negligible. On complex reasoning, open-ended generation, and multi-step tasks, it isn’t — accuracy drops precisely in the cases where a wrong answer is hardest to catch by eye, because it still reads as fluent and complete. That’s not an argument against tiering. It’s the argument for building the stakes axis in on purpose, instead of discovering it the way an entire industry just did, by accident, at scale.

GitClear tracked 211 million lines of code across four years of AI-assisted development and found the shape of that same failure, without anyone deciding to under-tier a single task. 2024 was the first year in their dataset that copy-pasted code exceeded moved-and-refactored code — duplicated code blocks up eightfold since 2021, two-week code churn up from 5.5% to 7.9%, and cloned code blocks carrying 15 to 50% more defects than original code. Nobody chose a cheap model for a hard problem and watched it fail on the spot. The volume of AI-generated output simply outpaced the review capacity that was supposed to catch what a faster, less careful pass would miss. It’s the same silent-erosion pattern from The System Around the Algorithm, one layer down: it doesn’t fail loudly, it accumulates quietly until someone finally audits the codebase and finds the bill.

That’s what the stakes axis is actually for. Not just deciding what earns the expensive tier up front — deciding what still needs a second set of eyes, a test, or a review step, regardless of which tier produced it.

This isn’t a new problem. Cloud computing already had it.

Overprovisioning compute for workloads that didn’t need it is the exact discipline the FinOps movement spent the better part of a decade fixing in cloud infrastructure — rightsizing instances, killing idle capacity, matching resources to actual demand. It was working: Flexera’s State of the Cloud research had tracked cloud waste declining for five straight years. Their 2026 report found that trend reversed — wasted spend ticked back up to 29%, and Flexera attributes the increase specifically to the rapid adoption of AI workloads outpacing the cost-governance practices that had been reining in waste everywhere else. The discipline that fixed this for cloud compute exists. It just hasn’t been applied to model selection yet, at most companies.

The rule I actually run

I keep a versioned policy file that maps task type to model and effort tier — not a rule of thumb I apply inconsistently, an actual table, changed only through a review step, that every part of my own system references by version instead of copying inline. Deterministic code handles what doesn’t need a model at all. Small, cheap models handle routine classification and extraction. Mid-tier models handle standard drafting and synthesis. Frontier and reasoning-tier models are reserved for what genuinely requires them.

What that looks like in practice: a task class like typo fixes or log-field extraction maps to the cheapest deterministic or small-model tier, no exceptions and no debate. Standard drafting and routine code review map to a mid-tier model. Architecture decisions and cross-service migrations map to frontier or reasoning tier, full stop, regardless of how routine the request sounds when someone types it into a chat window. And separately from all of that, layered on top rather than baked into the tier itself: anything capital-touching or irreversible gets routed through an evaluation gate no matter which tier wrote the first draft, because tier answers the cost question, not the stakes question. This is also where the loaded-cost argument I made in The ROI Number in Your AI Business Case Is Probably Wrong actually gets enforced, not just stated — tiering is the lever that keeps the denominator of that equation honest, deployment by deployment, instead of once at launch.

What I’d actually ask

Not “which model should we standardize on.” That question assumes one answer is supposed to cover every task, which is the assumption that got Uber and Microsoft to the same place from opposite directions. The better questions:

• Is there a written policy mapping task type to model tier, or is every engineer individually deciding, task by task, with no default but “the best one”?
• Who owns changes to that policy, and does a change require review, the way a change to any other cost-bearing config would?
• For the tasks currently on the most expensive tier — which of them are there because they need to be, and which are there because nobody built the cheaper path yet?
• When the cheap tier gets a call wrong, what actually catches it — a test, a reviewer, a second model — or does it just ship?

The biggest model isn’t the safe default. It’s the default that costs the most before anyone notices it was a default at all.

Part of a larger framework — The System Around the Algorithm