Best LLM Gateway for Production (2026): LiteLLM vs Portkey vs OpenRouter vs Requesty

On May 29, 2026, Palo Alto Networks closed its acquisition of Portkey, making it the core AI Gateway for Prisma AIRS. For production, pick Portkey for Prisma AIRS guardrails, LiteLLM for self-hosting, OpenRouter for paid and free models, or Requesty for per-key caps, fallbacks and an EU gateway. Requesty also has Enterprise team budgets. We make Requesty. For the two-way decision, see LiteLLM vs OpenRouter.

The best LLM gateway for production in 2026

An LLM gateway sits between your application and model providers. It gives your application a common entry point and applies configured rules to requests. You still choose which providers to trust, which models meet your quality bar and what happens when the gateway itself is unavailable.

The table compares deployment ownership, failure handling, billing and governance. Vendor documentation and pricing were checked October 2026.

All four support configured fallback. The useful test is what the fallback returns when your primary route fails, including its conversation state, location and answer quality.

Which gateway fits your startup stage?

Stage is only a starting point. The constraint matters more than headcount: a small healthcare startup with a private deployment requirement has a different decision from a larger team with a public chatbot.

Going direct to one provider is a perfectly good production choice when that provider covers what the application needs. A gateway adds another service to operate or depend on, so its controls should justify that dependency.

LiteLLM vs Portkey vs OpenRouter vs Requesty

LiteLLM: control your own deployment

LiteLLM is a Python SDK and self-hosted gateway. Its reliability documentation describes retries followed by automatic fallback to another model group. Its license is MIT outside the separately licensed enterprise directory. Pick it when you need to run the gateway yourself and can assign someone the proxy, secrets, upgrades and incident response.

On October 21, 2026, LiteLLM disclosed a compromised 1.107.0 release: malicious PyPI wheels and public container images contained credential-stealing malware. The maintainer says the affected artifacts were removed or repointed and its private enterprise registry was unaffected. If you installed an affected wheel or ran an affected image, follow the incident guidance to isolate and rebuild systems and rotate reachable credentials. Do not start by running pip in the affected environment.

Pin image digests to identify what you deployed; review provenance and incident notices to assess whether an artifact was compromised. For the corrected attack chain, see LiteLLM's later investigation.

Portkey: guardrails with hosting options

Portkey has a hosted platform and a self-hosted gateway. Its hosted Production plan lists partner and LLM guardrails, role-based access control, service-account keys and simple caching. Enterprise lists SSO, granular budgets and private-cloud deployment. The self-hosted package has its own feature set.

Portkey's pricing page lists semantic caching on Production, while its caching documentation limits it to select Enterprise plans. Its gateway repository labels Gateway 2.0 a pre-release.

Portkey's Developer plan includes 10,000 recorded logs per month and three days of log retention. Requests beyond the allowance continue without additional recorded logs. Production includes 100,000 recorded logs per month and 30 days of log retention, with $9 overages per additional 100,000 requests.

OpenRouter: managed access with production controls

OpenRouter combines a managed marketplace of paid and free models with provider selection. It can route among providers of the same model, while an explicit model fallback list tries other models. For a given model, provider selection can sort providers by price, throughput or latency.

OpenRouter's Auto Router classifies the task and selects from models ranked by the community's task-specific spend over the previous seven days, subject to your model restrictions and cost tier. It prefers the previous model in a conversation while it remains among the top candidates, but it can choose another model on a later turn. The Free plan does not include Auto Router; the separate free-model router selects among eligible free models. OpenRouter charges no additional Auto Router fee; you pay the selected model's rate and applicable credit-purchase fees.

A fallback to a paid model after a free model incurs the paid model's charges. OpenRouter also documents budgets, allowlists and guardrails, subject to plan. Its fallback rules include moderation errors, so decide which refusals should terminate a request instead of sending the same prompt to another model.

Requesty: managed routing and spend controls

Requesty puts a capped key, a customer-defined fallback chain and usage logs in front of a production workload. It provides an OpenAI-compatible API and an Anthropic-compatible API for 600+ models. You choose the models in a fallback chain, set retries and call the policy by name. Weighted load balancing and latency routing are separate policies.

A fallback reacts to failure; it does not judge task difficulty. On October 27, Requesty launched Auto Routing in beta for that decision. You describe routes and choose a model for each; a decision model reads the first user message and selects a route if it meets your confidence threshold, or uses your configured fallback. Include the decision model in your privacy and residency review: it receives the first user message. The choice stays pinned for 60 minutes per conversation. Send a stable trace_id for the session so Requesty can associate its turns. Each decision is billed as a request to the selected decision model.

Spend limits include monthly per-key caps, as well as user and organization limits. Groups require Enterprise. Their default limit applies per member; support enables Group Budget mode for a shared team budget. Enterprise includes SSO and audit logs, and the key management API is an Enterprise feature.

Requesty's auto caching uses provider prompt caching. Portkey's semantic response cache is a different mechanism. Portkey says semantic cache matching ignores the system prompt, so test cache namespaces and policy changes before enabling it.

A provider outage and a gateway outage are different

Suppose your primary model returns a 429. A configured fallback can try another model or provider while the gateway remains reachable. Now suppose the gateway itself cannot be reached. Its internal fallback chain cannot receive the request. If your availability target requires it, your application needs an independently authenticated path to a second gateway or directly to a provider. See fallback policies and load balancing for configuration details.

Conversation state adds another constraint. OpenAI documents both manually supplied history and provider-managed conversation state. Keep a transcript your application can send to an approved backup, and test tool-call formatting and structured outputs on both routes. An OpenAI previous_response_id identifies OpenAI-managed state; send the transcript when switching to another provider. Provider-specific state or encrypted reasoning may not transfer unchanged.

A new provider can also start with a cold prompt cache. OpenAI's caching documentation explains that a cache hit needs a matching prefix on a machine holding the entry. Measure the fallback's cost and time to first token.

Streaming needs two tests: a failure before the first chunk and a failure after partial output reaches the user. The OpenAI Python SDK explains why stream consumption is not automatically retried: replay could duplicate output. For an agent, a repeated tool call could also repeat an external action. Give side effects idempotency protection and define how the user sees an interrupted answer.

Compare total cost, not percentages alone

As of October 2026, Requesty's pay-as-you-go prepaid credits carry a 5% fee on top of credit top-ups, with no seat fees or minimums. OpenRouter's Standard plan charges 5.5% on credit purchases; Business charges 8%.

Assume you buy $100 in prepaid credits. Requesty's billing guide gives a $105 total and $100 credited to the balance: $100 + ($100 x 0.05) = $105. At OpenRouter Standard's published 5.5% fee, $100 + ($100 x 0.055) = $105.50, before any separate payment-specific terms. Each service deducts request costs from credits later. BYOK and post-paid terms are separate.

Bring your own key (BYOK) changes the bill. With OpenRouter BYOK, the provider bills inference directly. Standard and Business include $25,000 of list-price inference per month without an OpenRouter BYOK fee; above that, OpenRouter charges 5% of the equivalent list price from credits.

On Requesty standard plans, enabled guardrail scanners can add usage charges; Enterprise includes them. The console displays each scanner's price. Add provider charges, infrastructure, log retention, retries, cache behavior and backup model prices to the comparison. A cheaper primary model is no bargain if it takes more attempts to produce an accepted result. As another managed option, Vercel AI Gateway lists no token markup or platform fee; payment processing and optional services have separate terms.

Production readiness checklist

Use this checklist for any gateway. It gives the person approving the launch a record of the failure and policy decisions. Our LLM observability guide covers what to measure and alert on.

An alert reports a threshold; a cap enforces a limit. OpenRouter excludes BYOK spend from guardrail and workspace budgets by default; enable its inclusion separately for each policy if the cap must cover provider-billed usage. Prioritized BYOK keys default to shared-capacity fallback, which spends OpenRouter credits. If a particular provider account must handle the request, disable shared-capacity fallback for the relevant provider and restrict the permitted providers and fallback models. Other providers can otherwise still serve the request. Test the resulting configuration.

A 429 is not always a provider problem. Requesty's in-flight limit controls concurrent requests at the gateway, not requests per minute. The limit is shared across the organization; for non-enterprise prepaid accounts it varies with balance and tier. Check peak concurrency and auto top-up before launch. Changing the upstream model will not remove a gateway-wide concurrency bottleneck.

Residency has two hops

A gateway in Europe and a model in Europe are separate choices. Requesty's EU endpoint places Requesty processing in Frankfurt. Select EU-hosted model endpoints too, and include every fallback in the approved list. Organization admins can restrict which Requesty gateway regions are allowed. Our two-layer EU routing guide covers the gateway and inference decisions in more detail.

OpenRouter's regional routing is available on Business and Enterprise. OpenRouter's Auto Router is unavailable on EU and US regional domains, so regional workloads need eligible model endpoints and explicit routing instead. Its Europe region includes the EU and EEA, UK and Switzerland. If no eligible shared endpoint can serve a regional request, OpenRouter returns an error instead of selecting an out-of-region shared endpoint. Its documentation also describes customer-controlled BYOK resources and region declarations. Review the provider deployment, retention terms and data processing agreement.

Test a gateway before you commit

Set pass criteria before the trial: acceptable answer quality, maximum response time, permitted locations, maximum cost per completed task and who receives an alert. Use representative prompts, synthetic staging data and separate credentials.

1. 1
   Choose the workload and allowed routes

   Select primary and fallback models that support your context length, tools and output format. Include multi-turn conversations and the largest requests your app expects. Decide which provider errors justify a retry and which should stop the request.

   Build a repeatable evaluation set from your workload with expected results or a review rubric. Run it against the primary and each fallback, recording task success, schema and tool-call failures, latency and cost per accepted result. Keep the test set and model or policy versions with the release decision, then rerun them when you change a route.

2. 2
   Inject upstream failures

   Where the gateway supports a controlled test upstream, configure a provider-compatible test server as the primary and a healthy, distinguishable backup. Make the test server reachable from the gateway. Use synthetic data, dummy credentials and restricted ingress; Portkey SaaS requires a publicly reachable custom host. Set the application deadline long enough for the upstream timeout, configured retries and fallback attempt.

   Establish a healthy baseline for each target by temporarily making it primary, then restore the intended order. Reset any cooldown state between tests. Start with zero or low retries if supported, then repeat with your production retry settings. Return a provider-shaped 429 or 503 for every primary attempt, then repeat with the server withholding response headers beyond the configured upstream timeout. Record attempts, serving target, elapsed time and logs; compare them with your policy and application deadline.

   LiteLLM supports custom OpenAI-compatible endpoints. Its reliability guide says proxy mock flags have no effect, so use a real error response from the staging upstream. For a managed gateway, ask for an approved sandbox or fault hook if your test server cannot be used. Errors injected between the app and gateway test the app, not the gateway's upstream fallback. Never overload a shared provider or revoke production credentials to produce a failure.

3. 3
   Test the gateway path and streaming

   Use a staging network proxy or firewall to block the application's path to the gateway, then verify the independent backup. For streaming, interrupt once before the first chunk and once after the client has received content. A controlled upstream can flush a valid content delta, wait for the client to acknowledge it, then abort the connection. Check the completion marker, partial answer and any tool side effects; define how your app handles an incomplete stream.

4. 4
   Exercise the cap and inspect the evidence

   Use a low staging budget to test enforcement and alerts, including BYOK usage if applicable. Check that attempts appear in logs, the serving model and location match the policy, and the fallback answer passes your task evaluation. Measure time to first token, end-to-end latency and cost per accepted result on your own requests. Review retry budgets in both the client SDK and gateway so delays do not accumulate unnoticed.

Set up a production fallback on Requesty

Start with one workload, a capped key and a fallback policy. The policy below is customer-created, so create it before running the example.

1. 1
   Create a key and cap

   Sign up for Requesty, top up your balance if either policy target is a paid model, then create a key and set its monthly cap in API Keys. For alerts, configure a webhook and an API-key threshold in Spending Alerts. Put the key in the REQUESTY_API_KEY environment variable. Install the OpenAI Python SDK with pip install openai.

2. 2
   Create the fallback policy

   In Routing Policies, create a Fallback policy named production-fallback. Add a primary and at least one approved backup with compatible context, tool and region requirements. For example, the model library lists anthropic/claude-sonnet-5-5; choose it only if its endpoint meets your workload and location requirements. Set retries for each target and save the policy. The fallback guide describes the setup.

3. 3
   Call the policy and inspect usage

   Python

   Copy

   import os
   from openai import OpenAI
    
   client = OpenAI(
       api_key=os.environ["REQUESTY_API_KEY"],
       base_url="https://router.requesty.ai/v1",
   )
    
   response = client.chat.completions.create(
       model="policy/production-fallback",
       messages=[{
           "role": "user",
           "content": "Summarize this support ticket: Login fails after a password reset.",
       }],
       extra_body={"requesty": {"trace_id": "ticket-123"}},
   )
   print(response.choices[0].message.content)

   Check usage analytics and request logs, then run the failure tests above. See request metadata for trace IDs.

For an EU-only workload, use https://router.eu.requesty.ai/v1 and select EU-hosted models for every target. The EU routing guide covers both decisions.

So what should you pick?

For a prototype, pick OpenRouter to explore paid and free models through one marketplace. With your first paying customers, pick Requesty if a managed gateway, per-key cap and configured fallback solve your immediate problem. When enterprise deals require input and output checks inside Prisma AIRS, pick Portkey and test its guardrail behavior on your chosen endpoints. If a customer requires self-hosting, pick LiteLLM and assign an infrastructure owner.

This week, run one customer workflow through the staging tests above. Keep the route only after you have seen the backup answer, the cap stop traffic and the logs show which model served the request.