LiteLLM Alternatives (2026): 10 Managed and Open-Source Options

On October 21, 2026, a compromised LiteLLM release exposed a question for teams running its proxy: who should operate the gateway? Pick Bifrost for a self-hosted alternative, or Agent Router for an Envoy platform. For a managed gateway, compare Requesty, OpenRouter, Portkey, Cloudflare and Vercel. We make Requesty, one of the products compared here.

LiteLLM alternatives at a glance

This comparison covers 10 alternatives and LiteLLM as a reference. Portkey appears once: its open-source gateway and hosted service are options from the same vendor. Product and pricing information was checked October 2026.

A model server and a gateway do different jobs. A gateway routes and governs requests; the model runs at the selected inference endpoint. Self-hosting the gateway does not keep a prompt local when the configured model provider is external. An observability tool can also sit alongside a gateway for traces and evaluations; check whether it routes requests before treating it as a proxy replacement.

Why move from LiteLLM Proxy?

To be fair, LiteLLM does a lot well. Its open-source core gives teams a Python SDK and a centralized proxy, with broad provider coverage, virtual keys, spend tracking and routing. Its routing options include weighted, latency-based and cost-based strategies. If your team needs that control and is comfortable operating it, staying is a valid choice.

The operational question changes as the proxy becomes shared infrastructure. LiteLLM's quickstart starts a basic proxy without a database; virtual keys, spend tracking and the admin UI need a database. Its production guide recommends Redis for shared state when running multiple instances and covers database connections, migrations and upgrades. The bill for self-hosting includes that work, even when the software license costs nothing.

Security is the other reason teams are rethinking the setup. On October 21, 2026, LiteLLM published a compromised 1.107.0 package and affected container images. The maintainer's incident notice identifies the affected PyPI and container artifacts and says they were removed or repointed. Its October 23 investigation traces the attack to an exposed, unrevoked Agent Platform admin key and a stolen GitHub CLI token. Our incident response article provides background; its early account of the attack chain and floating container tags has been superseded. Use the maintainer's notice for affected artifacts and remediation, and its investigation for the final attack chain.

Separately, Wiz surveyed 3,074 publicly visible LiteLLM instances in February 2026 and found 9.6% accepted the example master key sk-1234 or required no authentication. That is a result for exposed instances, not every LiteLLM deployment. Our exposure analysis discusses the gateway security boundary, but its headline and opening conflate example-key acceptance with no-auth access. Wiz's count of 294 includes 191 instances with no authentication, so it combines both categories.

Managed hosting transfers gateway patching and infrastructure operations to a vendor. It also adds that vendor to your data and credential trust boundary. Review its security practices and provider contracts alongside your own access policies.

Open-source alternatives you can run

Bifrost: a Go-based proxy

Bifrost's Apache 2.0 gateway supports provider fallbacks and load balancing. Its virtual keys have budgets and request and token limits. Clustering and SSO are enterprise features; infrastructure and model calls still cost money.

Now, the coverage tradeoff: LiteLLM lists 100+ providers, while Bifrost lists 23+. Those are vendor counts; compare each upstream, custom endpoint and authentication method in your inventory.

Pick Bifrost for a self-hosted migration with tools for LiteLLM users. Its compatibility plugin transforms requests and responses. A separate migration tool reads a running LiteLLM proxy and its config, requires administrative access, and recreates supported entities. Run its dry run and review skipped providers and fields. Virtual-key token values are not migrated, so callers need new keys. Test and configure fallbacks separately.

For applications using the LiteLLM Python SDK directly, Bifrost also documents a LiteLLM SDK integration at /litellm; it works with providers supported by both products. Test existing calls before switching.

For a local evaluation, the Bifrost setup guide gives these commands:

export BIFROST_SETUP_TOKEN=$(openssl rand -hex 32)
npx -y @maximhq/bifrost

Open http://localhost:8080. Under Workspace, Config and Security, enable dashboard password protection and create an admin using the setup token. Add an OpenAI provider key with access to gpt-6-sol. A fresh instance has no dashboard password; keep it local while you set up authentication. The guide lists SQLite configuration and log files, so plan for persistent state and upgrades in production.

In a second terminal, set BIFROST_USER to the admin username. With inference authentication enabled, this documented Basic Auth method prompts for the password. The provider format uses openai/ followed by the OpenAI model ID:

curl http://localhost:8080/v1/chat/completions \
  -u "$BIFROST_USER" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-6-sol",
    "messages": [{"role": "user", "content": "Hello, Bifrost!"}]
  }'

This example uses Basic Auth without virtual-key enforcement. If you enforce virtual keys on inference, Bifrost documents sending an allowed virtual key in x-bf-vk alongside Basic Auth. Dashboard authentication and caller budgets are separate controls.

Portkey OSS: a gateway with a hosted path

Portkey's MIT-licensed gateway core provides routing and fallback. Pick it when you want to run the core yourself and retain a hosted path from the same vendor. Compare the core with the hosted plans for logging and governance.

Kong: build on an existing API gateway

Kong's open-source gateway runs on your infrastructure. Its basic AI Proxy plugin is in the open-source codebase. Pick Kong when your organization already operates it. AI Proxy Advanced requires an AI license.

Kong also has a managed option. Its Dedicated Cloud Gateways announcement says Kong operates both data and control planes and supports Kong AI Gateway.

Agent Router: Envoy and Kubernetes

Agent Router, formerly Envoy AI Gateway, is an Apache 2.0 project built on Envoy. Pick it for a platform team already working with Envoy or Kubernetes. The team owns deployment, upgrades and upstream provider configuration.

AISIX: a Rust gateway

API7's AISIX is an Apache 2.0 gateway with an OpenAI-compatible API. A declarative deployment needs no database or control plane. The OSS gateway includes MCP access, but budgets and spend caps belong to AISIX Enterprise or Cloud. Even with Cloud's hosted control plane, the customer runs the gateway data plane.

Check MCP separately from model routing

An MCP gateway connects clients to tools and servers; an LLM gateway routes model requests. If you use both, compare the products at the tool boundary as well as the model boundary.

Registering the same server on a new gateway does not transfer its credentials or prove that clients receive the same tool permissions. Include a real tool call and an unauthorized-tool test in the migration.

Managed alternatives to LiteLLM

Requesty: managed routing and controls

Point your OpenAI-compatible client at https://router.requesty.ai/v1, create a Requesty key and replace the LiteLLM alias with a model ID or a configured policy. You get routing and spend controls without operating the proxy or its database. The migration example below walks through the setup.

Requesty puts one endpoint in front of 600+ models. You configure fallback chains, weighted load balancing, spend limits and model access lists. Usage analytics track costs and performance across requests.

Requesty adds a 5% fee on top of credit top-ups on pay as you go, with no seat fees or minimums. Per-key monthly spend limits are documented in the API limits guide. Groups are an Enterprise feature: default Global mode applies a group limit to each member, while a shared team cap requires support to enable Group Budget mode. The free tier covers free models; paid models require a positive balance. BYOK is supported for listed providers. Pick an open-source option above if self-hosting is mandatory.

OpenRouter: a hosted model marketplace

Pick OpenRouter when you want one hosted API for discovering and accessing models across providers. Its default provider routing considers measured effective price, throughput, latency, rate limits, reliability and tool-call quality for the requested model. Set sort to order providers by price, throughput or latency; explicit sorting bypasses the default routing behavior. Pricing lists 5.5% on Standard credit purchases and 8% on Business. BYOK has separate allowance and fee rules.

What does a top-up cost? Assume a $100 credit purchase. On Requesty, 5% of $100 is $5, so the charge is $105. On OpenRouter Standard, 5.5% of $100 is $5.50, so the charge is $105.50. This compares the credit-purchase fees in Requesty's billing guide and OpenRouter's pricing; provider prices, taxes and BYOK terms are separate.

Portkey cloud: hosted or hybrid

Portkey cloud combines routing and observability. Its pricing page lists 10,000 recorded logs per month and three days of log retention on Developer. Requests beyond the log allowance continue, but the additional logs are not recorded.

Portkey's hybrid architecture runs the data plane in your VPC while Portkey hosts the control plane for administration, configuration and analytics. The data plane still needs an operator, and external model calls still reach their providers. Palo Alto Networks completed its acquisition of Portkey in May 2026.

Cloudflare: gateway at the edge

Pick Cloudflare AI Gateway if your application already runs on Cloudflare. It has caching, analytics and rate limiting. Its dynamic routing documentation describes conditions, percentage splits and fallbacks.

Cloudflare's pricing documentation lists free core features and a 5% fee on Unified Billing credit purchases. Inference, some logs and guardrail inference have separate charges.

Vercel: AI SDK and application workflows

Vercel AI Gateway is managed and does not require your application to run on Vercel. Pick it if your team already builds with the Vercel AI SDK. Its pricing documentation states no token markup or token platform fee, with separate payment-processing and optional-feature charges.

Vercel's BYOK documentation says spend through your own provider credentials does not count toward gateway budgets. Set provider-side limits too. The documentation also says a failed BYOK call can retry with system credentials and charge the gateway credit balance.

TrueFoundry: compare the deployment and fee model

Pick TrueFoundry for a comparison across SaaS, hybrid and self-hosted gateway deployments. Its deployment documentation describes each option.

Pricing lists Pro at $25 per user per month, including 20,000 gateway requests and tool calls per user per month. It lists $20 per additional 100,000 requests; model-provider charges are separate.

How to migrate from LiteLLM Proxy

An OpenAI-compatible client needs one base URL edit to point elsewhere. A production migration also needs a new credential, an accepted model identifier and equivalent controls. For Requesty, adapt direct LiteLLM Python SDK calls to a supported client. Bifrost also documents a LiteLLM SDK integration; test the existing calls and configuration before switching.

Map the alias before the endpoint

In LiteLLM's proxy configuration, model_name is a user-facing alias and litellm_params.model selects an upstream model. The alias support-chat could hide several deployments or a fallback chain. A Requesty model ID uses provider/model; a configured policy uses policy/name. Check the destination gateway's naming rules before changing callers.

Recreate a fallback chain

Assume a LiteLLM proxy has two NVIDIA NIM deployments. This configuration adapts the documented model alias, NVIDIA NIM prefix and fallback syntax. The aliases are examples; the proxy needs an NVIDIA key and access to both models.

model_list:
  - model_name: support-chat
    litellm_params:
      model: nvidia_nim/nvidia/nemotron-3-ultra-550b-a55b
      api_key: os.environ/NVIDIA_NIM_API_KEY
  - model_name: support-backup
    litellm_params:
      model: nvidia_nim/nvidia/nemotron-3-super-120b-a12b
      api_key: os.environ/NVIDIA_NIM_API_KEY
 
router_settings:
  fallbacks:
    - support-chat: [support-backup]

The client sends model="support-chat". LiteLLM targets Ultra and then the support-backup model group when its regular fallback is triggered. For Requesty, go to Routing Policies, select Create Policy and the Fallback strategy, and name it support-chat. Add nvidia/nemotron-3-ultra-550b-a55b first and nvidia/nemotron-3-super-120b-a12b second. Set retries, save, and call model="policy/support-chat".

Requesty tries the configured targets in order and returns an error if all fail. A direct model call bypasses the chain. LiteLLM has separate regular, context-window and content-policy fallback settings; test the target gateway's triggers against each behavior you use. Both example targets use NVIDIA, so they do not provide provider diversity. For an independent failure path, select an allowed model on a different provider and evaluate its output and data policy.

Both targets are free models. For a new organization, the 200 daily free-model requests are shared across free models; exhausting that allowance affects both targets. To cover that failure, select an allowed paid model on a different provider and fund your Requesty account before testing.

The same model name does not prove the two gateways use an identical backend deployment or provider account. Requesty's June 10 changelog names both NVIDIA IDs; confirm their availability and prices for your key before creating the policy. This example selects Requesty-managed NVIDIA access. NVIDIA is absent from Requesty's supported BYOK provider list, so the existing NVIDIA key and its provider quota do not transfer to this policy. Review location and retention for every fallback target.

For load balancing, configure weights and pass a stable trace ID if a conversation needs to stay on the same model. In the OpenAI Python client, add extra_body={"requesty": {"trace_id": "session-123"}} to the request. See the load balancing guide. Requesty's concurrency limits do not replace upstream requests-per-minute or tokens-per-minute quotas.

Make the first Requesty call

Create a Requesty account and key, install the OpenAI Python SDK with pip install openai, and export the key as REQUESTY_API_KEY. Requesty's free-model guide lists this model as free; check its price and availability in the model library before calling it. Paid models require a positive balance. The client follows the OpenAI SDK integration.

import os
from openai import OpenAI
 
client = OpenAI(
    base_url="https://router.requesty.ai/v1",
    api_key=os.environ["REQUESTY_API_KEY"],
)
 
response = client.chat.completions.create(
    model="nvidia/nemotron-3-super-120b-a12b",
    messages=[{"role": "user", "content": "Reply with a short greeting."}],
)
print(response.choices[0].message.content)

The base URL is the one-line endpoint change; the key and model also change from the LiteLLM setup. After creating the policy, replace the model with policy/support-chat to call the fallback chain, which attempts the primary target first. Induce a provider error in staging to verify that the backup is attempted.

Before choosing an ID, list the chat models available to your key:

curl https://router.requesty.ai/v1/models \
  -H "Authorization: Bearer $REQUESTY_API_KEY"

The model-list endpoint filters results by the organization's approved models and the key's access list. It lists chat models; use /v1/models/all for all modalities.

For the Anthropic SDK, the base URL is https://router.requesty.ai, without /v1. The Claude Agent SDK guide documents ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN and model selection for that integration.

Test the failure path, then shift traffic

A successful text response proves the basic connection. It says little about a production workload that streams output, invokes tools or expects a specific JSON schema. Use non-sensitive representative requests and test each behavior before moving traffic.

Check where a 429 originated before retrying: it could come from a provider or a gateway control. Replaying a request can also repeat a tool action, so use test data and avoid replaying side effects or personal data without authorization.

Review the replacement key's payload logging setting and the upstream provider's retention policy separately. Requesty request logs can include message content; turn logging off per key or use Enterprise zero data retention (ZDR) to stop new payload storage across the organization. When payload logging is on and ZDR is off, Requesty stores full raw request bodies separately in encrypted object storage; logged messages and responses are truncated. Enabling ZDR leaves existing payload logs in place, and the upstream provider's retention policy remains separate.

Choose the operating model first

Stay on LiteLLM if it works for your workload and your team can patch and operate it. Move to Bifrost if you want to keep self-hosting and prefer its local setup and migration tool; you still own the production gateway. Move to Requesty, OpenRouter or Portkey cloud if you want a vendor to run it.

This week, inventory one production alias: record its upstream models, keys, budget and fallback order. Recreate that path on your chosen gateway and run the failure tests before switching traffic. Start with the Requesty quickstart if a managed gateway fits.