Best AI Gateways in 2026: 10 Compared for Routing LLM Traffic

As of October 2026, pick LiteLLM or Bifrost for self-hosting, Portkey for guardrails, OpenRouter for model access, Requesty for managed routing with spend caps per key and team plus an EU gateway, Kong for API platforms, Cloudflare for edge, Vercel for AI SDK, TrueFoundry for model serving and Helicone for observability. Requesty publishes this guide and makes one of the products compared.

AI gateway comparison: 10 options for 2026

Prices and product documentation were checked October 2026. Gateway fees and model inference are separate cost components. Purchased credits can pay for managed inference; BYOK and self-hosted deployments can involve separate provider bills.

How we compared AI gateways

We picked ten gateways that teams use for LLM traffic and compared their documented deployment options, routing controls, observability, governance and pricing. The order starts with prominent self-hosted and widely searched products. The recommendations are editorial judgments based on vendor documentation.

An LLM gateway sits between an application and model providers. Depending on the product, it authenticates requests, routes them to a model, records usage and enforces policy. A general API gateway handles broader HTTP traffic; an AI gateway adds model-aware controls such as token accounting or provider-specific routing. Kong is an example of a general gateway platform with AI capabilities.

A common endpoint reduces integration work, but an OpenAI-compatible API does not promise that every model supports the same tools, streaming behavior or structured outputs. Test the request your application sends.

LiteLLM

LiteLLM is the obvious pick when you want to run a central model proxy on your own infrastructure. Its open-source repository documents a Python SDK and proxy, provider integrations, virtual keys, budgets, retries and load balancing. It also lists Auto Routing, so its routing options extend beyond a fixed fallback order.

The open-source gateway is free to self-host. LiteLLM prices Enterprise by gateway scale, with deployment and support affecting the quote. You still pay for provider inference and the infrastructure and people that operate the proxy.

Self-hosting also puts artifact security and credential rotation on your team. LiteLLM's March 2026 security advisory identifies two compromised PyPI releases and says official proxy Docker images were not affected. Pin and verify artifacts for any gateway you run.

Portkey

Portkey fits teams that want a gateway with routing, observability and guardrails, with hosted and private deployment options. Portkey guardrails include input and output checks, prompt-injection checks and deny actions, with availability varying by plan. Requesty's scanners focus on PII and secrets in request content. Portkey's streaming output checks are informational after the stream completes.

Its fallback documentation describes ordered and nested strategies. By default, a non-2xx response triggers fallback, and the status conditions are configurable. A policy that retries every error has different consequences from one restricted to provider outages.

Portkey's pricing page lists 10,000 recorded logs per month on Developer. Requests above that limit continue, but their logs are not recorded. Production lists 100,000 recorded logs per month and $9 per additional 100,000 requests. Enterprise lists SSO and private cloud deployment. Ask for the current Production subscription price and your required retention.

The gateway core is MIT licensed; hosted Enterprise controls have separate plan terms. Palo Alto Networks completed its acquisition of Portkey in May 2026. Portkey's current docs also use the name PRISMA AIRS AI Gateway.

OpenRouter

OpenRouter is a hosted model marketplace and API for teams that want to explore providers and consolidate model access. Its fallback API accepts ordered backup models. A fallback from a free model to a paid one incurs the served model's price. OpenRouter also documents Anthropic Messages support, so evaluate the interface your client needs.

The Standard plan charges 5.5% when credits are purchased; Business charges 8%. Those are credit-purchase fees, not per-token markups. For bring your own key (BYOK), Standard and Business include $25,000 per month in OpenRouter list-price inference without an OpenRouter fee. Above that allowance, OpenRouter charges 5% of the equivalent list price from credits, while the provider bills inference directly.

OpenRouter is hosted. For regional traffic, Business and Enterprise add in-region endpoints, with a separate responsibility for customer-controlled BYOK deployments discussed below.

Bifrost by Maxim

Bifrost is a Go-based alternative for teams that want an open-source gateway in their own environment. Its repository documents fallbacks, semantic caching and virtual-key budgets. Bifrost's pricing page lists a free open-source edition and custom Enterprise pricing. Enterprise includes features such as adaptive load balancing, cluster mode, SSO and audit logs.

Evaluate Bifrost alongside LiteLLM if language, deployment footprint or the split between open-source and paid features affects your platform decision. You own the runtime and should price its infrastructure, monitoring and on-call work alongside the license. Measure latency on your own traffic.

Kong AI Gateway

Kong AI Gateway fits an organization already using Kong for API traffic. Its AI Gateway product page describes routing, semantic caching, quotas, guardrails and observability for AI traffic. The AI Proxy plugin handles provider authentication and normalizes supported request formats.

The attraction is a shared platform for existing APIs and AI endpoints. The work is understanding which plugin and edition supplies each control, then configuring them for the model providers you use. Kong's pricing page lists commercial plans; get a quote for the AI Management features and deployment you need before comparing it with a per-token gateway bill.

Cloudflare AI Gateway

Cloudflare AI Gateway is a hosted option for teams using Cloudflare's edge stack. Its documentation lists retries, fallbacks, caching, rate limiting and analytics. Core gateway features are free; Unified Billing adds a 5% fee when credits are purchased, while provider inference is billed at passed-through rates.

Cloudflare resolves recognized credentials on the request first, then a stored default BYOK key, then Unified Billing if fallback is allowed. To prevent the last step for third-party providers, enable Require provider credentials, set byok_only: true, or use the documented per-request header.

Accounts that created their first gateway on or after September 24, 2026 use Workers Logs pricing and retention; accounts that created a gateway earlier use Legacy Logs. Workers Free includes 200,000 log events per day with three-day retention. Workers Paid includes 20 million per month, then $0.60 per additional million, with seven-day retention.

Vercel AI Gateway

Vercel AI Gateway is a hosted choice for AI SDK users, and it works from infrastructure outside Vercel too. Its gateway documentation describes AI SDK, OpenAI-compatible and Anthropic Messages interfaces. Vercel lists no token markup or platform fee on tokens, with payment processing and optional features potentially adding charges. The free tier includes $5 per month for eligible models. Once you purchase credits, the account moves to the paid tier and the monthly free credit no longer applies.

Vercel's budgets apply to gateway system-credential spend, not provider BYOK spend. Project budgets apply to project deployment OIDC traffic, not arbitrary traffic using an API key. Vercel's pricing documentation also describes fallback from failed BYOK credentials to system credentials. Put limits on the provider account as well as the gateway, and check which credential paid for the request.

TrueFoundry

TrueFoundry fits an enterprise combining model serving, an LLM gateway and MCP governance. Its product page describes those capabilities as one platform, with SaaS and Enterprise deployment options.

Developer is free for up to three users, with 10,000 requests and tool calls per user per month. Pro costs $25 per user per month and includes 20,000 per user per month; additional usage is $20 per 100,000 requests. Each client LLM request and MCP tool call counts, while retries and failed calls inside the gateway do not. Model-provider costs are separate and passed through without token markup. VPC, on-prem and data residency are Enterprise features.

Assume a one-user Pro account makes 120,000 counted calls in a month. The allowance is 20,000 and the remaining 100,000 adds $20. The gateway charge is $25 + $20 = $45, before provider inference. Count seats and MCP calls in your own forecast.

Helicone

Pick Helicone if gateway routing and observability belong in the same workflow. Its gateway documentation and repository describe routing and fallback alongside tracing, sessions and cost analysis. The platform has hosted and self-hostable options.

Helicone's pricing page lists a free allowance of 10,000 requests per month and usage-based plans. Its gateway documentation describes using credits with 0% markup. The pricing page says Helicone is no longer part of Mintlify; ask about its roadmap, paid retention and hosted versus self-hosted features before a long contract.

Requesty

Pick Requesty for managed model routing with spend controls and governance across teams. One API provides access to 600+ models, with fallback policies, weighted load balancing and latency routing among the models you select. Set monthly spend caps for API keys and users, with team and organization controls also available. Groups are Enterprise; support enables Group Budget mode for a shared team budget. Usage analytics groups cost by model, provider, user, key or custom metadata.

Auto Routing, launched in beta on October 27, lets you write route descriptions and assign a model to each. A decision model reads the first user message and selects a route if it meets your confidence threshold; otherwise it uses your fallback. The choice is pinned for 60 minutes per session. Send a stable trace_id per session when your client supports it. Each decision is billed as a request to the selected decision model.

Requesty also has PII and secrets scanning, approved models and an MCP Gateway. Guardrails scan request content; they do not block prompt injection or scan responses for newly generated PII or secrets. Standard-plan scanning has usage-based charges. Enterprise includes SSO, audit logs and role-scoped analytics and logs; on pay as you go, members see organization-wide analytics and logs. Requesty limits in-flight concurrency, not requests per minute.

The free tier includes 200 requests per day on free models. Pay as you go has a 5% fee on top of prepaid credit top-ups, with no seat fees or minimums; Enterprise pricing is custom. LiteLLM and Bifrost are better fits if you need to self-host, while Vercel lists no token markup.

Compare the billing meter

Assume a $1,000 credit purchase fully spent on managed inference, with the same model rates in each case. Exclude taxes, provider discounts, caching, retries, BYOK and optional features.

The arithmetic is $1,000 multiplied by 1.05 for Requesty and Cloudflare, 1.055 for OpenRouter Standard, and 1.08 for OpenRouter Business. Vercel's listed token price gives $1,000 before processing and optional charges.

Use the linked pricing pages for Requesty, Cloudflare, OpenRouter and Vercel. Then add the features you intend to use. For self-hosted products, add compute, storage, support and the cost of operating the gateway. Free software does not make inference free.

Choose by deployment and workload

For coding agents, a chat completion alone is a weak trial. Run a real task with streaming, tool calls and a long conversation. Check which key identifies the developer, what happens when a model fails and whether the fallback can handle the same tool schema. LiteLLM documents agent integrations, and Bifrost describes coding-agent governance. Requesty documents Claude Code setup; Vercel documents coding-agent access in its gateway guide.

Compare enterprise security controls

A Gray Swan Cygnal-only rule forwards text-only streams without a Cygnal check. If it detects a supported tool call, it holds subsequent chunks; earlier text remains visible.

Portkey's input checks evaluate the last message, and synchronous mode is needed to gate a request. Bifrost's guardrail documentation says a provider error or timeout fails open. Test scanner failures and streaming behavior alongside the checks themselves. Private gateway deployment also leaves the location of model inference and any external scanner to verify.

EU data residency has more than one layer

A gateway endpoint and a model deployment are separate choices. Requesty's EU endpoint processes gateway traffic in Frankfurt. It does not force inference into the EU: choose EU-hosted models and restrict the organization's allowed gateway regions.

Review organization approvals and every key, group and group-default access list, along with all fallback targets and BYOK deployments. Use only vetted EU deployments in each list, restrict who can change or attach lists, and test with non-sensitive data that non-EU models are rejected for each relevant key and fallback path. Include keys with multiple groups and keys with explicit lists in the test.

OpenRouter's in-region routing is available on Business and Enterprise. For shared provider endpoints, the regional domain fails closed when no eligible endpoint can serve the request. The regional domain does not move or verify your own BYOK deployment. OpenRouter's Europe region also includes the UK and Switzerland, so check the exact location if your policy requires EU-only processing. Account records, tool destinations and contracts need their own review.

For both products, inspect model regions, log storage, subprocessors and the DPA. A regional URL alone cannot satisfy every GDPR obligation.

Test your shortlist before production

A gateway demo proves that one request works. Your production trial should prove what happens when a request does not work, who pays for it and what data leaves your boundary.

1. 1
   Pick the workload and controls

   Choose representative prompts, tools and output schemas. Record acceptable task quality, latency, monthly budget, allowed providers and data regions. Give every candidate the same workload and service tier.

2. 2
   Force a failure

   Force a primary-provider 429 and record the final provider, request result and charge. Pass if the configured backup serves the request inside the allowed region. For a separate partial-stream test, interrupt the stream after a test tool action has been recorded, then exercise your application's retry or fallback path. Reuse the same idempotency key for the replayed action and verify it is recorded once. Adapt the test to the tool's idempotency mechanism.

3. 3
   Reconcile the bill and data path

   Compare gateway logs with provider invoices. Record the credential, inference region and data storage used for the successful request.

A provider-hosted conversation ID is a particular risk. OpenAI's conversation-state guide describes chaining with previous_response_id or sending previous messages manually. A different provider cannot be assumed to understand that ID. Keep a portable application-owned transcript if cross-provider continuation is required, then transform it to the backup model's supported format. Test tool calls and partial responses separately.

A provider switch also changes the cache boundary. OpenAI's prompt caching guide describes matching prefixes and cache scope; measure the backup as a cold-cache case. The original provider's cached prefix does not automatically follow the conversation.

Run a Requesty fallback policy

If a managed gateway fits your shortlist, create a Requesty account, create an API key and set it as REQUESTY_API_KEY. Add credits if your policy calls paid models. Follow the fallback policy guide to create a policy named my-fallback, select compatible models and set their order and retry settings.

For Claude Code, the integration guide uses the Anthropic-compatible base URL without a /v1 suffix. With Claude Code installed, run:

export ANTHROPIC_BASE_URL="https://router.requesty.ai"
export ANTHROPIC_AUTH_TOKEN="$REQUESTY_API_KEY"
export ANTHROPIC_MODEL="policy/my-fallback"
claude

Will your team run the gateway itself? Pick LiteLLM. Need a managed gateway with governance across teams? Pick Requesty. This week, force a 429 on a representative coding task and check the backup's tool calls, region and bill before rolling it out.