Introducing Auto Routing: your rules, the right model

Recently we pointed some of our own Claude Code sessions at a routing policy instead of at Opus 5.5. A third of their calls ended up on Haiku, about a sixth on Sonnet, and only half needed Opus. The model bill came out 42% lower than the same tokens would have cost at Opus 5.5 prices.

That policy runs on Auto Routing, which is live in beta in the Requesty Console from today.

Auto routing is the thing customers ask us for again and again. Leo from our product team says as much at the start of the video above. The pitch writes itself: send any prompt and the right model gets picked, cheap when the task is easy and the strongest one when it is hard. So why did it take us this long?

Why auto routing is harder than it sounds

Three problems kept coming back every time we sketched it.

The first is that two prompts can look the same and hide ten times the work. "Add a field to the Invoice struct" and "add a field to the Invoice struct and make sure every handler, the streaming path and the export job agree on it" open with the same seven words. One is a two-line change. The other touches half the codebase.

The second is that difficulty is relative. A Terraform change is routine for a platform team and a minefield for a frontend team. A router that ships with one global idea of what counts as hard will get it wrong for somebody, and that somebody has no way to correct it.

The third is the one people underestimate. Switching models halfway through a conversation throws away the prompt cache, and on an agent the cache is usually the biggest lever on the bill. Some quick numbers at today's list prices: reading 100,000 cached tokens on Opus 5.5 costs $0.02. Moving that same context to Sonnet 5.5 mid-session means writing it into a fresh cache at $2.50 per million tokens, so $0.25. The "cheaper" model costs more than twelve times as much for that one turn, before it has written a single token. A router that re-picks on every message can quite easily cost you more than it saves.

So most routers end up guessing, message by message, using a definition of difficulty that somebody else wrote. When the guess is wrong you pay for it, either in money or in quality.

So we flipped the question

You know your tasks, your codebase and your priorities better than we do, and better than any lab or gateway does. So with Auto Routing you decide which kind of work goes to which model, and we make the call in a fraction of a second.

An Auto policy has four parts:

• A question. A sentence or two of context about your users or your code, ending with what you want to know, such as "What kind of change is this?"
• Routes. Each one is a short plain-language label and a target. The target can be a model, or one of your existing fallback, load balancing or latency policies.
• A decision model that scores every route against the first message.
• A fallback and a threshold. If no route clears the threshold, or the decision model errors, the request goes to the fallback. We suggest starting the threshold at 50% and making the fallback the model you trust most.

There is no training data, no embeddings and no code to write. If you can describe the split in a sentence, you can route on it: the size of a code change, support triage, reasoning depth, language, topic or risk.

What happens to a request

You set the model to policy/<your-policy-name> and nothing else in your code changes. Then:

1. The decision model reads the first user message only. Not the system prompt, not later turns, not tool results.
2. It receives your question as a multiple choice, with your route labels as the options, and returns a probability for each one.
3. The top route wins if it clears the threshold. If none does, the fallback takes the request. Either way the request goes through, because the decision model never blocks.
4. The choice is pinned for the session. Later turns skip the decision model and stay on the same model for 60 minutes, keyed on your trace_id, so prompt caching keeps working.

The decision runs once per conversation and adds one short call at the start of a session, typically under a second.

Why a decision model and not an LLM

We could have asked a small LLM to pick a route and parse its answer. We went with decision models instead, which we have spent the past few weeks benchmarking on 22,000 decisions. Models like TypeSafe's Jev and Cloudflare's Clef do not write text at all. They answer a typed question with a probability for every option in a fixed set. That is exactly the shape of a routing decision, and it gives you a number you can put a threshold on, rather than a label an LLM happened to print.

They are also cheap. Jev costs $0.042 per million input tokens and nothing for output. Claude Code prepends your project instructions and system reminders to the first message, so call it 10,000 tokens. Classifying that costs about four hundredths of a cent, once per session.

You choose the decision model per policy, from the ones your org has approved. Jev did best on coding prompts in our tests. Clef is fast and, with its smaller sibling Clef Flash, the only option that reads images. Microsoft's Decision-1 runs on Azure, including in EU regions. Each decision is billed as a normal request to that model and shows up in your logs next to the call it routed.

Test it before you ship

Every policy has a Try it panel. You paste a first message, run it against several decision models at once, and see where each one lands: the consensus route, plus every model's probability for every route plotted against your threshold. Only the decision models run, never the target models, so a test costs a fraction of a cent. You can run it on the Global router or on your org's regional router, such as the EU one.

In the video we tried this on a couple of prompts. Ask about changing a button colour and every decision model agrees. Ask about a backend rewrite and they split, and since Jev is not confident, the policy plays it safe and sends the request to the fallback. That split is the most useful thing the panel shows you, because it points straight at the labels that overlap.

A few things we learned writing our own policies:

• Describe the request, not the model. The decision model never sees model names, so "needs a strong model" means nothing to it.
• Keep labels distinct. Two labels that overlap split the probability between them, and both can end up under the threshold. If two routes point at the same model, merge them.
• Give brief context, then ask. Long instructions and worked examples tend to overfit.
• Test with real first messages from your logs, including whatever system reminders your client adds to them.

What happened when we ran it on ourselves

Back to that run. The policy we used is our own router for coding agents working on our Go backend, claude-auto-coder:

The question describes the codebase in two sentences and asks the decision model what the agent will need to touch. It ends with a tie-break: when several existing places all have to change, prefer the last label. Refactors go to Opus even though they change no behaviour, because a rename has to land in every place at once.

Across the Claude Code sessions we sent through that policy, 51% of calls went to Opus, 16% to Sonnet and 33% to Haiku. Compared with sending every one of those calls to Opus 5.5:

• Model cost was 42% lower for the same tokens.
• Model time was 30% lower, decision overhead included.

How can the saving be that large when half the calls still went to Opus? Because the gap between the tiers is wide. Haiku 5.5 lists at $0.10 per million input tokens and $0.50 per million output tokens. Opus 5.5 lists at $4 and $20. That is a factor of 40, and Sonnet 5.5 sits at half of Opus on both. If every call carried the same number of tokens, our mix alone would land at roughly 40% cheaper, which is close to what we saw.

Put another way, at this mix every $100 your agents spend on Opus today would come to about $58. That is plain arithmetic on our mix, not a forecast, and your mix will look different.

How we measured, because it matters:

• Cost is the same tokens repriced at Opus 5.5 rates, so both sides cover the same work.
• Model time is modelled, not clocked. We took platform median latencies per model, stream mode and input size, scaled them to the time we measured, and included the decision model's overhead.
• Quality is not graded here. This benchmark does not score whether Opus would have written better code on the calls that went to Haiku. We tested that separately: on 129 real tasks from our Go monorepo, a decision model on the first message matched Opus 5.5 on tasks solved at 72% of the cost. That post has the full method and where the approach stops working.
• Scope. This is one internal benchmark on one codebase. It is not independently verified, and results will vary with your workload and your policy.

Using it in Claude Code

Point Claude Code at Requesty and set the model to your policy:

export ANTHROPIC_BASE_URL=https://router.requesty.ai
export ANTHROPIC_AUTH_TOKEN=<your Requesty API key>
export ANTHROPIC_MODEL=policy/claude-auto-coder
claude

From code, the policy is just a model name. Pass a stable trace_id for each session so every turn stays on the same decision:

import os
from openai import OpenAI
 
client = OpenAI(base_url="https://router.requesty.ai/v1", api_key=os.environ["REQUESTY_API_KEY"])
 
response = client.chat.completions.create(
    model="policy/claude-auto-coder",
    messages=[{"role": "user", "content": "Add a created_at field to the Invoice struct"}],
    extra_body={"requesty": {"trace_id": "session-8f2c"}},
)
print(response.model)  # the model the route picked

Auto policies work with the Chat Completions, Messages and Responses APIs, including streaming and tools. The full reference is in the Auto Routing docs.

So what?

Not every task needs the biggest model, and most teams already know which of theirs do. What was missing was a way to write that knowledge down once, have it applied to every session, keep the cache warm while doing it, and never depend on somebody else's idea of hard.

If you want a starting point, the Console has three templates:

• Coding, by size of the change. Small local edits go to a fast model, and changes that must stay consistent across many places go to the strongest one.
• Support triage. Simple account questions go to a cheap model, and bugs or escalations go to a stronger one.
• Reasoning depth. Chit-chat and lookups stay fast, and multi-step reasoning, maths and planning escalate.

A few things to know while this is in beta:

• The first message decides. A session that starts small and grows into something big stays on the model it started on. Start a new session, or send a new trace_id, to get a fresh decision.
• Images are only read by Clef and Clef Flash.
• Auto policies cannot point at other Auto policies. They can point at your fallback, load balancing and latency policies.
• Only org admins can create or edit policies, and every target model has to be approved for your org.

We are curious to see the routes people write. The split that works for our Go backend will not be the one that works for your codebase, and that is rather the point.

You can set up your first policy in the Requesty Console. Side effects may include smaller invoices and warmer caches.