Best LLM Routers Compared (2026): Requesty, Portkey, LiteLLM, OpenRouter and More

On October 27, 2026, we launched Requesty Auto Routing in beta. For the best LLM router, pick LiteLLM for self-hosting, OpenRouter for a managed marketplace, Portkey for metadata rules, or Requesty for plain-language routes, a decision model, session pinning, fallback and load balancing. Requesty makes one of the products compared.

In our Claude Code sessions, model cost was 42% lower than repricing the same tokens at Opus 5.5 rates. That comparison did not grade task quality or establish classifier charges.

Product details were checked October 2026.

The best LLM routers at a glance

As of October 2026, the best fit depends on what the router reads and how long it remembers a decision.

A selector and a gateway can share the job. The selector decides which model should answer; the gateway can then provide credentials, logging and a fallback path. Combining layers also creates more settings and failure modes to test.

What an LLM router chooses

An LLM router chooses a model or endpoint according to a policy. Some routers forward requests; others return a recommendation for the application or gateway to execute. The word "routing" covers two different decisions.

Model selection for the task

A support request asking how to reset a password and a report of a complex account failure have different requirements. Requesty classifies the first message against categories and models you choose. Not Diamond has learned selectors, while OpenRouter Auto ranks candidates using task-specific aggregate spending.

A routing decision is only as good as what the router gets to read. A classifier that sees only the first user message cannot judge a complication revealed by a later tool call. A rule that sees the customer's account tier knows that tier, but does not automatically assess the task's difficulty.

Provider selection for reliability

After choosing a model, a gateway can choose an eligible deployment, distribute traffic by weight or retry another target after an error. A fallback can help when an endpoint returns a rate-limit or server error. A successful call still needs a quality check.

A fallback model also needs to support the request. Check tool calls, streaming, structured output, context size, input type and region for every target in the chain. Requesty's fallback guidance calls out compatibility with request parameters.

A request or a whole session?

A per-turn selector can react when a conversation changes. A session-pinned selector avoids routine model switches, which helps preserve cache affinity, but the first choice can become a poor fit if the work gets harder. Switching provider or model can mean writing a fresh prompt cache.

Requesty Auto Routing pins a route for 60 minutes per conversation. LiteLLM's session affinity is opt-in and off by default; an explicit escalation can raise the model tier. OpenRouter Auto can change its model choice as the conversation progresses.

Compare deployment, limits and pricing

This matrix covers the same shortlist. Model inference is separate unless stated otherwise. Self-hosted software also needs infrastructure and an operator. Prices and plan terms are as of October 2026.

Requesty: customer-defined routing for a session

Requesty provides managed access to 600+ models, with routing policies, spend controls and usage analytics. Auto Routing adds model selection to the existing fallback, weighted load-balancing and latency policies.

You write a question and distinct route labels, then assign each label a model or an existing reliability policy. A decision model reads the first user message, scores the labels and selects the highest-scoring route if it reaches your threshold. Low confidence or a classifier error sends the request to your configured fallback. That fallback still depends on its own target being available. The Auto Routing documentation describes the sequence and setup.

The classifier sees the first user message, the question and route labels. It does not see separate system messages, later turns or tool outputs. Text injected by a client into the first user message is included. For image requests, choose a classifier that reads images: the current docs identify Clef and Clef Flash for that job.

The choice is pinned using a stable trace_id when supplied, or conversation recognition otherwise. A target that is itself a fallback or load-balancing policy can still select a different underlying endpoint. If later work needs escalation, design that escalation explicitly. A new session or trace_id starts a new decision.

Set up a support router

1. 1
   Create the policy

   An organization admin follows the Auto Routing setup to create support-router. Write a question such as "What kind of support request is this?" Add distinct labels for account questions, technical bugs and escalations, then attach an approved model or reliability policy to each.

2. 2
   Choose the decision model and fallback

   Pick an approved classifier, set the fallback to a model you trust with difficult requests, and start with the documented 50% threshold. Adjust it using labeled examples. Classifier decisions are billed as normal requests to the chosen decision model.

3. 3
   Fund and test the policy

   Create a Requesty API key in the console and top up your prepaid balance before testing if your classifier or target models are paid. The free plan covers free models. An organization admin manages billing. Use the Console's Try it panel with representative first messages; it calls the classifiers without calling the target models. Save the policy and evaluate the complete workflow.

Install the OpenAI Python SDK and set your Requesty key:

pip install openai
export REQUESTY_API_KEY="your-requesty-api-key"

Call the saved policy:

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["REQUESTY_API_KEY"],
    base_url="https://router.requesty.ai/v1",
)
 
response = client.chat.completions.create(
    model="policy/support-router",
    messages=[{"role": "user", "content": "How do I reset my password?"}],
    extra_body={"requesty": {"trace_id": "support-session-001"}},
)
print(response.model, response.choices[0].message.content)

Reuse the trace ID for turns in this conversation and generate a different one for each new conversation. support-router is the policy you created, not a built-in model ID. The setup guide also shows how to use a policy with Claude Code.

Routing needs an outcome test

Our separate coding experiment evaluated task outcomes. Use your own tasks to choose the labels and threshold.

For reliability, Requesty also supports ordered fallback, weighted load balancing and latency routing. Its auto caching handles provider-side prompt caching for supported providers. Cache savings apply to eligible reused input, not the entire bill.

LiteLLM: self-hosted model selection

LiteLLM is the stronger choice when you want to operate the proxy yourself and change its routing logic. Its beta Auto Router classifies requests using heuristics, an LLM, Jev, keyword rules or a custom plugin. You configure the model targets for each tier. Optional session affinity pins a model name, with escalation supported; deployment affinity is a separate setting. Beta settings can change across releases.

Its reliability router separately supports weighted, latency, usage and cost-based deployment strategies. Picking the lowest-cost configured deployment is a different decision from assessing whether that model meets your quality bar.

The open-source proxy has no license fee to self-host. You still pay for infrastructure, model calls and the work of operating and upgrading it. Self-hosting the gateway alone does not keep inference local: the selected model endpoint must also meet that requirement. LiteLLM pricing describes its enterprise options.

OpenRouter: managed model access and automatic selection

OpenRouter is a good fit when you want a managed marketplace and a ready-made selector. Its Auto Router classifies the task, ranks models by aggregate spending for that task over the trailing seven days, then applies the selected cost tier and restrictions. It reranks each turn and prefers the previous model only while it remains a top candidate. Spending is a market signal, so validate its choices on your own tasks. Auto has no additional router-specific fee; selected model charges and platform terms still apply.

The separate Jev Router chooses a model and reasoning effort from the conversation. Its models and allowed_models lists are preferences that can be relaxed when no listed candidate is routable; use excluded_models for hard exclusions.

OpenRouter also selects providers for a chosen model and supports explicit provider sorting. Its regional routing excludes multi-model router aliases such as Auto. Customers provision and verify the region of their own BYOK deployments separately.

Portkey: routing from application metadata

Portkey is useful when the application already knows the rule: for example, a customer tier or a URL path determines the target. Its conditional routing evaluates conditions in order and can combine them with fallback and load balancing. A missing referenced key or malformed condition evaluates false. Metadata keys must be flat, and parameter comparisons support primitive values. Test every named branch and the default.

Portkey has a hosted platform and an open-source gateway. Its pricing page describes a free Developer tier with 10,000 recorded logs per month. Requests beyond that log allowance still run, but additional logs are not recorded. Palo Alto Networks completed its acquisition of Portkey in May 2026.

More routing options

Bifrost, TrueFoundry and Cloudflare

Bifrost's Complexity Router uses a decision model or semantic reference matching to classify the latest user request. Optional session-aware routing keeps the highest tier reached and allows escalation. That retains a tier, not necessarily the serving provider, key or cache entry. Its self-hosted OSS gateway is free; model calls and operations are separate.

TrueFoundry Auto Routing maps simple, medium and complex tiers to configured targets. Later user turns can raise the tier, and a fallback can change the target. A Responses API follow-up using previous_response_id stays on the original backend. Cross-target fallback is disabled, though configured retries against that target still apply. TrueFoundry's default heuristic runs in process without a model call; LLM and decision-model classifier calls incur their own charges.

Cloudflare Auto Router classifies the conversation and scores eligible models for expected quality and cost. With a session ID it normally retains a model for a turn; fallback can select another eligible model. Between turns it weighs the benefit of switching against the cost of losing cache. Dynamic Routing is a separate way to configure conditions, weighted splits and fallbacks.

Inworld, DigitalOcean and Braintrust

Inworld evaluates customer-written conditions in order. Its traffic splitting can keep a variant sticky for a user. A variant can use Auto ranking by price, latency or other metrics; a sticky Auto variant does not guarantee a fixed underlying model. Its price ranking adds listed input and output token rates, so calculate cost using your own token mix.

DigitalOcean Inference Router selects a model per request and lets you bias the choice toward budget or quality. It uses cache-aware routing and reports held and switched requests, but the router alias does not pin a specific model. During public preview, there is no additional router fee and the serving model is billed. Personalization against your evaluation rubric is support-assisted during preview.

Braintrust selects an eligible provider for a model named in the request. Its evaluation tools help teams score outputs and inform model policies; online scoring runs asynchronously. The application or a separate routing layer must enforce a quality-based model choice. The hosted gateway is free during public preview; model usage and evaluation charges are separate.

Weave, Not Diamond and RouteLLM

Weave Router makes a model and provider decision for each agent API request. Its self-hosted default uses an on-box cluster scorer and enables session pinning. The hosted service uses a different production routing model and factors cache-switching cost into its decisions. It can switch models as a session changes. Requesty instead pins its first-message route for 60 minutes. Pick Weave when per-action coding-agent selection fits your workflow; self-hosting means operating the router and its database.

Code Boost costs $100 per seat per month and includes $200 in monthly usage, subject to the same allowance windows. For paid fallback, Weave charges 5.5% on credit top-ups, with an $0.80 minimum. BYOK inference is billed by the provider, and Weave debits a 5% fee on provider spend from the credit balance.

Not Diamond supplies pretrained and custom model selectors that integrate with an existing gateway. Its coding router uses session history and cache state to recommend a model and reasoning effort for each step. Its coding router documentation describes an early preview. Its pricing lists $0.05 per million tokens routed; the selected model's inference is separate.

RouteLLM is an open-source framework for selecting between a designated strong model and a weak model using configurable thresholds. It is useful for teams that want to inspect and evaluate routing methods. Two documented routers, mf and sw_ranking, require an OpenAI key for embeddings, even when the target models are elsewhere.

Kong and Helicone

Kong AI Gateway fits teams already using Kong. Its semantic balancer embeds a prompt and matches it to customer-configured target descriptions using a vector database. Kong also documents load balancing and failover. The self-hosted AI Proxy Advanced plugin requires its AI Gateway Enterprise offering.

Helicone's provider routing starts with a caller-selected model and chooses among its providers. Its open-source gateway also documents model-latency routing across configured models and supports self-hosting. Choose Helicone when provider reliability and request analysis are the immediate jobs, or evaluate its open-source gateway if you want to operate the routing layer.

Compare router costs on the same basis

Requesty's pay-as-you-go prepaid plan has a 5% fee on top of credit top-ups, with no seat fees or minimums. A $100 top-up costs $105 and adds $100 in credits. Those credits then pay for usage; do not add the fee a second time to each request. Requesty's free plan covers free models and 200 requests per day.

Assume you buy $100 in Requesty credits, pay $105, and spend all $100 on model and classifier usage. The fee is $100 x 0.05 = $5. If you spend only $50 during the period, cash paid remains $105 and $50 in credits remains. Compare cash paid, usage and the remaining balance, especially when top-ups and usage fall in different months.

For the same $100 credit purchase, Cloudflare Unified Billing also charges $105. Assume the same underlying model rates and full credit use; logging, classifier and other optional charges can differ.

OpenRouter Standard charges 5.5% on credit purchases; Business charges 8%. On Standard and Business, OpenRouter includes $25,000 per month of list-price BYOK inference without an OpenRouter fee. Above that, it charges 5% of its list value, deducted from credits. The provider bills the inference separately. Prepaid credit purchases and BYOK usage have different billing terms.

The more useful denominator is a completed task. Add the bill for successful and failed attempts, including classifier calls, retries and cache reads or writes. Divide by the number of tasks that passed your acceptance test. A cheaper call that needs a second attempt can cost more per accepted result.

Test routing on your own tasks

Start with representative tasks and set a quality bar before the comparison. Define a scorer for each task category: use exact checks for structured outputs and a documented rubric for open-ended answers. Calibrate any model judge against human-reviewed examples. Split examples used to tune routes from a locked evaluation set. Run the same locked tasks through your current model, a lower-cost model and the proposed router. Include complete multi-turn sessions for agent work, since the first message, later tool results and cache behavior affect the outcome.

Assume a sample of 100 tasks and a required 95% pass rate. The baseline passes 96 tasks at a total cost of $120. The routed setup passes 94 at a total cost of $80. The baseline costs $120 / 96 = $1.25 per passed task; the router costs $80 / 94 = about $0.85. The router is cheaper per pass but misses the trial quality bar. Change its routes or fallback and test again on a fresh locked set before shipping.

With a sample this small, inspect the paired task failures and uncertainty around the pass rates before drawing a production conclusion. Repeat the comparison on more representative tasks.

Retries deserve a separate test. The OpenAI Python SDK retries certain connection and HTTP errors twice by default. If the gateway also retries, the number of upstream attempts can grow. Set a shared deadline and inspect what happens after streaming has begun; replaying a partial response can duplicate output.

For EU workloads, include the classifier, target models and fallbacks in the residency review. Requesty's EU endpoint places Requesty's gateway processing in Frankfurt; the inference location is determined separately by each selected model endpoint. OpenRouter's regional documentation also distinguishes regional routing from the setup of BYOK resources.

Which router should you choose?

For a production app, set up reliability routing first: choose compatible fallback targets, configure load balancing where it helps, and test provider failures. Then add model-choice routing when you can measure quality for each route. A lower model bill is useful only if the work still passes.

Pick LiteLLM or Bifrost if owning the deployment is the priority. Pick OpenRouter for a managed marketplace with a market-based selector. Pick Portkey for explicit metadata rules, or Cloudflare for its Auto Router and rule engine. Pick Not Diamond for learned selection alongside a gateway, and RouteLLM for research on strong-versus-weak routing. Pick Kong if your team already runs Kong and wants semantic routing.

For coding agents, pick Weave when you want a selector that makes decisions per agent request. Pick Requesty when you want to write the route categories and pin the choice across a session. Pick DigitalOcean for managed task-aware selection without a fixed model pin, or Braintrust to pair provider routing with its evaluation workflow.

This week, collect a locked set of tasks, run your current model and one routing policy on the same set, and compare pass rate and cost per passed task. To set up customer-written routes, sign up for Requesty and follow the Auto Routing guide.