Best Managed LLM Gateway (2026): Hosted Options Compared

As of October 2026, pick Requesty for hosted model access and spend controls; OpenRouter for a marketplace; Vercel for no token markup; Cloudflare for Workers; Portkey for guardrails; Merge for customer budgets; TrueFoundry for agents; Kong for API management. Pick LiteLLM for self-hosting. We make Requesty.

Best managed LLM gateways compared

The shortlist prioritizes a vendor-operated request path, multi-model access, pricing and governance. Start with the required data region and provider account, then compare the fee meter and controls. Product and pricing details were checked in October 2026.

For a self-hosted reference, LiteLLM's open-source gateway has no software license fee. Its Enterprise edition adds a license and support to the same gateway image in your cloud. Pick it when open source and runtime control are worth operating the proxy yourself.

What fully managed means in practice

A gateway sits between your application and model providers. Its data plane receives requests and applies routing or access rules. Its control plane is where those rules are configured. A hosted dashboard alone does not tell you who runs the proxy: Kong's architecture distinguishes its hosted control plane from proxy nodes, while Portkey's hybrid deployment places the data plane in the customer's VPC.

For this guide, fully managed means the vendor runs the request-serving gateway and its operational dependencies. The service agreement should identify who handles scaling, patches, secret storage, monitoring and incidents, along with the uptime commitment. Optional features can introduce other services, as the Kong example below shows.

Your team still chooses which provider accounts to connect, which models employees can call, how much each key can spend and what goes into logs. A vendor-operated gateway also adds a service dependency. Provider fallback inside that gateway helps when a model endpoint fails; a gateway outage needs a separate path.

Do you need your own provider account?

A gateway key authenticates your application to the gateway. Which upstream credentials it uses is a separate choice. The table distinguishes a direct OpenAI connection from an OpenAI model reached through a reseller.

BYOK means bring your own key: your provider bills you for inference under that account. A failed key can cause a gateway to use its own credentials instead, changing both the invoice and the data path.

Which hosted gateway fits your team?

Model access and team controls

Pick OpenRouter for a hosted model marketplace with consolidated access. Its guardrails include budgets and model or provider restrictions. Business and Enterprise provide regional domains. BYOK inference has its own billing and deployment conditions.

Pick Requesty for a hosted endpoint to 600+ models, configured fallbacks, per-key spending caps and access controls. We run the gateway. You choose the models and fallback chain. Enterprise adds SSO and audit logs. Our guardrails scan inputs for PII and secrets; use another control for prompt injection or output moderation. Standard-plan prices are per million input tokens for each enabled scanner and depend on the action.

Pick Merge if your application needs policies and budgets per embedded customer. Its Build Your Own Router scores eligible models against customer-selected weighted benchmarks; it is not available on customer routing policies. Free and Pro organizations created on or after September 4, 2026 at 5:00 p.m. Eastern need an access grant for Anthropic models, including on Bedrock. Its BYOK documentation describes managed-credential fallback, so select organization credentials only if that boundary is required. Merge's published BYOK fee descriptions use different meters; ask for a sample invoice.

Test customer and project overrides against your organization policy. An authorized administrator can set a less restrictive override that applies when a key is pinned to that scope. Merge DLP forwards a request unscanned if the scanner fails or times out.

Your existing platform

Pick Vercel AI Gateway if your application uses the AI SDK or your main priority is zero token markup. It also works outside Vercel-hosted applications. The tradeoff is that gateway budgets exclude BYOK spend, so set a provider-side limit as well. Project budgets cover OIDC-authenticated deployments, but API key spend does not count toward the project budget. Set per-key budgets for API key traffic and a team budget for covered team-wide spend.

Team-wide provider allowlisting and team-wide zero-data-retention each have separate charges on eligible plans; Vercel also documents a free per-request ZDR option on Pro and Enterprise.

Pick Cloudflare AI Gateway if your team already uses Workers and Cloudflare controls. Its core features include analytics, caching and rate limiting. Cloudflare supports stored provider keys; Unified Billing is a separate option with a credit-purchase fee.

Guardrails, agents or an API estate

Pick Portkey Cloud when input and output checks, prompt management and hosted gateway operations belong in one product. Palo Alto Networks completed its acquisition of Portkey on May 29, 2026. Portkey guardrails evaluate only the last message in the request body. Streaming output checks run after the stream completes and cannot trigger a fallback or retry. Its Developer plan continues serving requests after its log allowance is exhausted, but does not record the extra logs. Portkey's pricing page lists the Production subscription; confirm its availability before budgeting for it.

Pick TrueFoundry SaaS if you want an LLM gateway alongside MCP and agent management. Its Pro plan includes 20,000 requests and tool calls per user per month, with model-provider costs separate. Its Managed Models use TrueFoundry capacity, currently in the United States.

Pick Kong if your organization already manages APIs through Konnect. Dedicated Cloud Gateways use Kong-operated data planes, and Kong's plugin compatibility table lists AI Proxy and AI governance plugins for cloud gateways. This is the plugin-based deployment; Konnect also has arrangements where you operate the proxy.

Some plugins need services of their own. Kong's PII Sanitizer uses an anonymizer service, and its semantic cache needs a vector database and an embeddings model. Those dependencies belong in the quote and operating plan.

Managed versus self-hosted: the cost of ownership

A self-hosted gateway needs engineering time for upgrades, secrets, monitoring and incidents, plus infrastructure. Hosted gateway charges depend on the service and enabled features. Your team still configures and tests it.

Take a monthly workload with $10,000 of identical model usage on both paths. Assume both paths need the same application, configuration and testing work. The engineering and infrastructure rows show only the extra work of operating the self-hosted proxy. Assume eight engineering hours per month at a loaded $100 per hour and $200 per month for infrastructure.

For Requesty, assume a $10,000 credit top-up is consumed in that month. Requesty adds a 5% fee on top of prepaid credit top-ups, so the fee is $500 and the cash paid for credits is $10,500.

The extra self-hosted operations total is (8 x $100) + $200 = $1,000 per month. The managed fee is $10,000 x 0.05 = $500. Under these assumptions the managed path costs $500 less per month. The comparison excludes self-hosted setup, paid licenses, incidents, taxes and other fees. It assumes the same provider rates, models and cache behavior.

The break-even point is $1,000 / 0.05 = $20,000 of monthly credit top-ups, assuming credits are consumed in that month. At a $40,000 top-up, the fee is $2,000, or $1,000 more than the assumed monthly proxy operations. If setup takes another 40 hours at $100 per hour, spread over 12 months, add about $333.33 per month. Break-even then rises to about $26,667 per month. Engineering time is an economic cost; it does not imply a company can immediately reduce payroll by that amount.

Compare gateway ownership costs

Replace the assumptions with your own. Everything is per month unless stated.

Monthly model spend and matching credit top-up ($)Incremental proxy engineering hours per monthLoaded engineering cost ($/hour)Incremental proxy infrastructure per month ($)One-time proxy setup hoursSetup amortization (months)Credit top-up fee (%)

Self-hosted total

$11,000

Managed total

$10,500

Monthly difference

$500

Cheaper

Managed

Break-even: $20,000 of monthly credit top-ups

Show the arithmetic

• Operations: 8 × $100 + $200 + 0 × $100 / 12 = $1,000
• Managed fee: $10,000 × 5% = $500
• Self-hosted: $10,000 + $1,000 = $11,000
• Managed: $10,000 + $500 = $10,500

A different fee meter changes the calculation. OpenRouter Standard charges 5.5% on credit purchases: buying $10,000 in credits incurs a $550 fee, for $10,550 in cash. Cloudflare Unified Billing charges 5% on credit purchases. Vercel has no token markup, but payment processing and enabled governance features can add charges. Compare invoices for the same workload and credit balance, including provider discounts, guardrails and logs.

Where do requests and logs run?

A regional gateway and a regional model are two separate choices. Requesty's EU endpoint processes gateway traffic in Frankfurt. A request sent to that endpoint can still select a model hosted elsewhere.

For an EU-only path, restrict the organization's Requesty serving region to EU and approve EU-hosted model deployments. Review every API-key access list and, if your Enterprise organization uses groups, every group and group-default access list. More specific lists take precedence in the documented resolution order. Include only eligible deployments in every routing and fallback policy. With harmless test prompts, check that representative keys reject models hosted outside the EU and that fallback stays in the approved region.

OpenRouter's regional domains use a different mechanism: OpenRouter filters shared endpoints by region. If no eligible shared, BYOK or private endpoint can serve the request, it returns an error. Customer-managed BYOK deployments must also be configured in the required region. Its European boundary includes the UK and Switzerland, so an EU-only requirement needs a closer review. Account records and export destinations have separate conditions.

For either service, trace the full path before a privacy review: gateway processing, provider inference, logs, backups, exports and support access. Check retention and the data processing agreement. On Requesty's self-service plans, prompt and output logging is on by default and can be disabled per API key. Requesty's data privacy documentation distinguishes payload logging from telemetry, which remains when payload logging is disabled. Requesty's DPA is available on request.

Test a hosted gateway before production

Run the same representative requests through your shortlisted services. Include tool calls, structured output, long inputs and streaming if your application uses them. Record task success and end-to-end latency as well as the invoice.

Use the OpenAI-compatible API guide to test interface differences and the AI governance template to assign control owners and retain test evidence.

Budget behavior deserves its own test. Vercel excludes BYOK spend from gateway budgets, while Cloudflare's spend limits cover BYOK for models with known pricing and are eventually consistent. Concurrent requests can pass a limit before every cost is recorded. Test the burst your application sends, and set upstream limits when needed.

For fallback, check the response as well as the HTTP status. A backup model can accept the request but produce a different schema or tool call. Also account for retries in both the SDK and the gateway: a timeout after an upstream request has begun can leave your application uncertain about what completed. Protect side-effecting tools against duplicate execution.

Keep separate credentials and a configured endpoint for the backup path. Set an end-to-end timeout and test that the application can reach the backup without calling the primary gateway.

Start with Requesty

We run the gateway; you choose the models and policy. Create a Requesty key, fund it and send a request to our OpenAI-compatible endpoint using the quickstart.

1. 1
   Create a key and add credits

   Sign up for Requesty, create an API key using the quickstart, and add credits in billing for the paid model in the next step. Set the key in your shell:

   Shell

   Copy

   export REQUESTY_API_KEY="your-requesty-api-key"

2. 2
   Send a request

   Shell

   Copy

   curl https://router.requesty.ai/v1/chat/completions \
     -H "Authorization: Bearer $REQUESTY_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"model":"openai/gpt-6.1-terra","messages":[{"role":"user","content":"Explain a managed LLM gateway."}]}'

   The Requesty model library lists available models.

3. 3
   Set a policy and limit

   Create an ordered fallback policy, then use its name as policy/your-policy-name in the model field. Set a monthly limit on the API key and test your backup models with the same request format.

For EU processing, use https://router.eu.requesty.ai/v1 as the OpenAI-compatible base URL. Apply the serving-region, access-list and fallback checks above before moving sensitive traffic.

So what?

Two questions decide the purchase. First, who will run the proxy? Pick LiteLLM if your team wants the open-source runtime and will operate it; pick a hosted gateway if you want the vendor to do that work.

Second, which constraint leads? Pick Requesty for hosted model access and spend controls, OpenRouter for a marketplace, Vercel for zero token markup, Cloudflare for Workers, Portkey for guardrails, Merge for customer budgets, TrueFoundry for agent management or Kong for an existing API estate. This week, take one workload through the pilot checklist and compare its bill and failure behavior with your current path.