Open Source AI Gateway vs Managed: Which to Choose in 2026

On October 19, 2026, Helicone said it was no longer part of Mintlify. As of October 2026, its gateway, LiteLLM, Bifrost and seven other open source AI gateway projects below are self-hostable. Pick self-hosting for deployment or code control; pick managed to avoid running the gateway. We make Requesty, the managed option in this comparison.

Best open source AI gateways to self-host in 2026

Pick LiteLLM or Bifrost for a standalone proxy, Kong or Apache APISIX if you already operate that gateway, and Agent Router or agentgateway for the Kubernetes or agent-protocol fit below. Pick Pangolin or OpenZiti when connecting private model servers is the main problem.

The license links cover the named repositories, not every commercial edition or dependency. Project details are current as of October 2026.

Which edition you run changes what you deploy. LiteLLM's Docker guide runs the proxy with Postgres. Portkey documents Node, Docker and other deployment paths. Bifrost's gateway quickstart says a fresh dashboard and admin API have no password. Keep the pilot private and follow its dashboard authentication instructions before exposing it.

Bifrost's virtual MCP documentation describes curated tool bundles as open source and requires governance to be enabled. Enterprise adds access profiles and other scoping controls.

Pangolin's site connector reaches a private model API over a tunnel. A private gateway resource controls client access separately; it can still route to a cloud model. Start with its Community Edition install and self-hosted model guide. The default install guide calls for a public IP, so it is not an air-gap recipe. Pangolin Community Edition uses AGPLv3: modifying the program and offering it over a network can carry source-sharing obligations. Review your use with counsel.

OpenZiti's gateway is an OpenAI-compatible chat proxy for OpenAI, Anthropic and compatible local backends. Its optional zrok overlay connects clients or backends privately. The documented provider routing uses fixed model prefixes, and the local backend implementation does not add an upstream bearer key. Enable gateway API-key authentication separately.

For Helicone, the self-deployment guide says to run the AI Gateway alongside Jawn for proxying. The gateway README calls the license Apache, while its LICENSE file says GPLv3. Its ownership notice says it is no longer part of Mintlify. Ask about support and the roadmap, and review GPLv3 distribution obligations with counsel.

Where does the prompt go?

An AI gateway sits between an application and model endpoints. It can authenticate the caller, apply routing rules and record usage. Where that software runs is separate from where the model runs.

Put LiteLLM on a private cluster and forward a request to a public model API: the gateway is inside your network, but the provider still receives the prompt. For an air-gapped workload, inference and supporting dependencies must also be inside the permitted boundary. Mirror images, packages and model weights, then test with outbound access blocked.

Map the control plane as well. That is where you configure keys, routes and policies; it can be separate from the data plane that handles requests. Record the locations of logs, backups, telemetry exporters and support access. For a managed service, review the gateway region and model deployment separately: our EU routing documentation explains the two layers.

Open source describes rights to use and change code under a license. Self-hosted describes who runs it and where. A managed service can use open source, and self-hosted software can have a commercial license. Neither label establishes where inference or logs live.

Open source versus managed: who owns the work?

Pick self-hosting when policy requires a disconnected gateway, a custom plugin is central to the product, or a platform team can own the service. Pick Agent Router over Requesty when the gateway itself must run in your Kubernetes environment.

Pick managed when its deployment and data terms meet your requirements and your team wants to focus on the application. You still own model selection, access policy, spend monitoring and your incident plan. A managed gateway adds a vendor dependency alongside the model provider.

There is a third option, and teams do take it: build your own router. That buys code-level control, but your team owns provider adapters, failure handling and policy changes. Build when configuring or extending an existing gateway cannot provide the behavior you need.

What does an AI gateway cost to own?

The free license leaves one question: how much engineering time will the gateway take to run? Assume the same workload, models, token usage, provider rates, caching outcome and output quality. For a 12-month plan, use $10,000 per month in underlying model usage. The self-hosted feature set has no license fee.

Requesty pay as you go adds a 5% fee on top of prepaid credit top-ups, with no seat fees or minimum spend. Assume Requesty-managed provider access, USD credits purchased and consumed in the same month, and no promotions; exclude BYOK, Enterprise, taxes, currency conversion and other transaction or optional feature charges.

Both sides need application integration and monitoring. Include separate security review, egress, support, enterprise licensing and incident costs if they apply to your quote. Model usage on the Requesty side is paid through Requesty, so do not add a second direct-provider charge.

Let S be monthly underlying model cost. With all other inputs fixed, self-hosting costs S + $2,300; Requesty costs 1.05 x S + $100. Set the two equal: ($2,300 - $100) / 0.05 = $44,000 in monthly model cost. At that point, each option totals $46,300 per month. Below it Requesty costs less in this scenario; above it self-hosting costs less.

Now, change the labor assumption and watch the answer move. At 5 hours a month, self-hosted non-model cost becomes $1,550 and break-even falls to $29,000: ($1,550 - $100) / 0.05. At 20 hours, non-model cost is $3,800 and break-even rises to $74,000: ($3,800 - $100) / 0.05. A model discount, different cache hit rate or paid OSS feature changes the result too. Compare the effective bill for each route.

Caching shifts the comparison. Requesty's provider-side prompt caching reuses an eligible prefix at the provider while the model still handles the request. Bifrost's response cache can replay a stored answer without a generation call. Semantic lookup still calls an embedding provider, adding its own cost and latency. Its direct and semantic modes both need a vector store; semantic mode also needs an embedding provider. Test tenant isolation, stale answers and answer quality before including response-cache savings in the model. Our guide to provider-native tools through a gateway covers other behavior to test.

Run a production trial before choosing

Test budgets, fallback and streaming with the same representative workload on each candidate. Record accepted task quality, latency, effective cost and the operations required to support it.

1. 1
   Define the boundary and feature set

   List approved model endpoints, regions, retention requirements and required features. Identify which are included in the exact license or plan. For an air gap, block egress and test dependencies, including updates and telemetry.

2. 2
   Deploy and secure a pilot

   For LiteLLM, review the Docker guide and its Compose file. Run export LITELLM_BIND=127.0.0.1: in the shell that starts Compose, and check the resolved port mapping before starting. Docker documents how shell values take precedence and how to inspect the resolved configuration. Replace the latest image with an approved release before starting. Verify artifact provenance, secure the master key and create a separate virtual key for the application. Keep LITELLM_SALT_KEY stable: the quickstart says regenerating it makes stored provider credentials unreadable.

3. 3
   Test controls and failures

   Send a streaming request with tools, exceed a small test budget, rotate a credential and make a primary provider unavailable. Verify the result, logs and alerts. Test a provider 429 too: our 429 troubleshooting guide explains how to diagnose it. Check restore and rollback before directing production traffic to the gateway.

One LiteLLM detail is easy to miss: its documented no-database path lacks virtual keys and spend tracking, and a configured global max_budget does not enforce a cap on that path. Use the database-backed deployment for those controls and prove the cap works with a test request.

Release provenance belongs in the trial too. LiteLLM's October 21 incident report describes compromised release artifacts. Verify source identity and artifact provenance alongside signatures, restrict credentials, and prepare a rebuild and key rotation plan.

Try Requesty as the managed option

Requesty runs the gateway for you and gives your application one endpoint for 600+ models. Configure fallback policies with the models you choose, set spend limits, and use Approved Models to restrict access. Use Access Lists to restrict a key's models; groups require Enterprise. If no access list or Approved Models policy is configured, the catalog is allowed by default.

Sign up and create a Requesty key, add prepaid credits for the paid model below, install the OpenAI Python SDK with pip install openai, and set REQUESTY_API_KEY in your environment. The OpenAI SDK setup uses this client and request shape. The example calls openai/gpt-6.1-terra, released October 26 and available on Requesty as of October 2026:

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["REQUESTY_API_KEY"],
    base_url="https://router.requesty.ai/v1",
)
 
response = client.chat.completions.create(
    model="openai/gpt-6.1-terra",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

For a LiteLLM migration, replace the key and model ID, recreate budgets and routing rules, and use the OpenAI compatibility checklist. Once you have created a fallback policy, reference it as policy/your-policy-name in the model field. Keep the previous route available for rollback.

Under the assumptions above, Requesty costs less below $44,000 in monthly model spend; self-hosting costs less above it. If your gateway must be air-gapped, pick a self-hosted option regardless of that threshold. This week, price your own operations hours and run a LiteLLM or Bifrost pilot, or send a paid-model request through Requesty and compare the two paths.