LiteLLM is an open-source AI gateway and Python SDK that gives applications one OpenAI-compatible API for calling LLM providers. As of November 2026, LiteLLM lists more than 140 providers. You can use the SDK inside a Python application or run the proxy as a gateway for applications and teams.
The software is free to use in production, but your team pays for the infrastructure, maintenance and security work needed to run the proxy. Using an operating estimate from TrueFoundry, a LiteLLM competitor, Requesty's 5% fee breaks even with self-hosting at $40,600 to $48,600 in monthly model spend.
How does LiteLLM work?
LiteLLM provides a common way to call different model providers. The SDK has a completion() interface, a consistent output format and a Router for retries and fallbacks. For shared use, the proxy sits between clients and providers and adds administrative controls.
| LiteLLM SDK | LiteLLM proxy | |
|---|---|---|
| Where it runs | Inside a Python application | As a self-hosted gateway |
| How applications call it | Through the Python interface | Through an OpenAI-compatible API |
| What it adds | A common completion interface, retries and fallbacks | Virtual keys, cost tracking and an admin UI |
A proxy request follows this path:
Application -> LiteLLM proxy -> Model provider
Application <- LiteLLM proxy <- Model providerThe proxy receives the application's request, calls the model provider and returns the response. An OpenAI-compatible client can point to the proxy endpoint. Our guide to LLM gateways explains where a gateway sits in an application.
The proxy can issue a virtual key to an application or person, complete with a budget and rate limit. LiteLLM shows cost by key, user, team and organization, and platform teams can set budgets and rate limits.
What does it take to run the proxy?
The Docker quickstart starts LiteLLM on port 4000 alongside PostgreSQL. It generates a master key for admin access and a salt key for stored credentials. Before production, it recommends pinning a tested release tag or digest.
The database belongs in the operating budget. LiteLLM's quickstart says virtual keys require a database, and so does litellm_settings.max_budget. Without a database, a max_budget: 100 setting keeps serving requests past $100 without an error or alert.
There is a scaling step too. LiteLLM's production guidance says spend tracking can become a PostgreSQL bottleneck at roughly 1,000 or more requests per second or 10 or more instances. It recommends a Redis transaction buffer for that case.
The LiteLLM quickstart says LITELLM_SALT_KEY has no in-place rotation: changing it makes every stored provider credential unreadable until you re-enter it.
Is LiteLLM free?
The gateway and SDK are open source and free to use. Model provider charges still apply. Your team also pays for the infrastructure and maintenance needed to run the proxy.
LiteLLM Enterprise is a paid license. The Enterprise documentation lists SSO and SCIM, audit logs, secret managers and support SLAs on top of the open-source gateway.
What might the monthly bill look like? TrueFoundry, a LiteLLM competitor, estimates $300 to $700 per month for infrastructure and $1,730 per month for DevOps time. That produces a total of $2,030 to $2,430 per month before model charges.
TrueFoundry also estimates 2 to 4 weeks of senior DevOps time for deployment and 10 to 20 hours per month for maintenance. These are its estimates: your bill depends on the infrastructure and staff time you use.
What happened in the 2026 security incidents?
Operating a gateway means owning its security updates and credentials. LiteLLM documented two supply chain compromises in 2026. A separate scan found exposed gateways with weak or absent admin authentication.
In March, LiteLLM said compromised PyPI versions 1.82.7 and 1.82.8 were available for about 40 minutes on March 24. The company traced the compromise to the Trivy dependency in its CI/CD security scanning workflow.
In a February scan reported in September, Wiz found 3,074 LiteLLM gateways on Shodan. Of those, 294 accepted sk-1234, including 191 that had no key set at all. That is about 9.6% of the February sample.
The report says the master key is both an admin credential and the switch that enables authentication. The Docker quickstart had dropped sk-1234 by September. We cover the finding in our LiteLLM default key exposure article.
Then, on October 21, LiteLLM reported that a malicious 1.107.0 package was live on PyPI from 11:39 to 13:23 UTC. Docker tags including v1.107.0 and latest were also compromised, from 11:46 to 14:20 UTC.
The malicious release contained a credential stealer. Microsoft's analysis says starting Python in an affected environment could trigger the payload without an application importing LiteLLM. The collector also tried to read credential and model configuration records from the proxy database.
A separate backdoor checked a GitHub file every hour for a domain and could run code fetched from it. The file held test values. Our account of the LiteLLM 1.107.0 incident covers the compromise in more detail.
These incidents create different jobs for the operator: securing the deployment, protecting keys and assessing package updates. Pinning a tested release is part of LiteLLM's production guidance. Your team still has to decide when and what to update.
When does a managed gateway make sense?
Self-hosting LiteLLM gives your team control of its proxy and the option to use the open-source software without a gateway license fee. A managed gateway shifts hosting and patching to the provider.
Requesty is a hosted LLM gateway with an OpenAI-compatible API. It connects to 600+ models, and there is no proxy for you to deploy or patch. Requesty charges 5% on top of model costs.
| Self-hosted LiteLLM | Requesty | |
|---|---|---|
| Hosting and patching | Your team hosts and patches the proxy | Requesty hosts and patches the gateway |
| License or fee | Open-source gateway is free; Enterprise is a paid license | 5% on top of model costs |
| Database and Redis | Your team operates PostgreSQL for virtual keys and litellm_settings.max_budget; Redis can be needed for shared limits or spend tracking at scale | No database or Redis for your team to operate |
| API compatibility | OpenAI-compatible proxy API | OpenAI-compatible API |
Take TrueFoundry's estimate of $2,030 to $2,430 per month for infrastructure and DevOps time. Treat this range as the self-hosted operating bill, with the same provider cost for the same model usage in both cases.
At Requesty's November 2026 fee of 5%, the lower end breaks even at $2,030 / 0.05 = $40,600 in monthly model costs. The upper end breaks even at $2,430 / 0.05 = $48,600.
This calculation compares monthly gateway costs. It leaves out TrueFoundry's estimated deployment time and any paid Enterprise license. Your own infrastructure and staff costs may put the break-even point elsewhere.
On these numbers, Requesty's fee is lower than the estimated operating bill below $40,600 in monthly model spend. Above $48,600, self-hosting can cost less if your team already has the capacity to run the proxy. Between those amounts, the result depends on the operating bill.
Plug in your own monthly costs to find your break-even point. Our open-source versus managed gateway guide covers the choice in more detail.
Before you choose a gateway
0 of 4 done
To try the managed route, see Requesty's AI gateway and compare the fee against your estimated operating bill.
Frequently asked questions
- What is LiteLLM?
- LiteLLM is an open-source AI gateway and Python SDK from Berrie AI. It connects applications to more than 140 LLM providers through an OpenAI-compatible API.
- What is the difference between the LiteLLM SDK and proxy?
- The LiteLLM SDK gives Python applications a common completion interface across providers. The self-hosted proxy is a gateway with virtual keys, cost tracking and an admin UI.
- Is LiteLLM free?
- The LiteLLM gateway and SDK are open source and free to use. Model provider charges and the cost of running the proxy are separate, and LiteLLM Enterprise is a paid license.
- How do you run LiteLLM with Docker?
- The LiteLLM Docker quickstart starts the proxy on port 4000 alongside PostgreSQL. It generates a master key and salt key, and recommends pinning a tested release tag or digest before production.
- Does Netflix use LiteLLM?
- LiteLLM lists Netflix as a user, alongside a testimonial from a Netflix staff engineer.
- OCT '26
LiteLLM 1.107.0 was backdoored: what happened, who is exposed and what to rotate
On 21 October 2026 the litellm 1.107.0 release on PyPI and the official Docker images shipped a credential stealer and a remote code loader. The payload runs every time Python starts, even if your code never imports litellm. Here is the full timeline in plain words, what the malware takes, how the attacker got into the release pipeline, a remediation checklist, and what this means for anyone running an AI gateway that holds every provider key.
- SEP '26
One in ten exposed LiteLLM gateways still answers to sk-1234: your AI gateway is a tier one security asset now
Wiz Research scanned 3,074 internet facing LiteLLM instances and found 9.6% accepting the example master key or no authentication at all. CISA put the MCP auth bypass on its exploited list with a 16 September patch deadline. The gateway that holds every provider key, reads every prompt and can reach your cloud metadata service is the most privileged box in your AI stack. Here is what the findings mean and how to run one safely.
- OCT '26
Open Source AI Gateway vs Managed: Which to Choose in 2026
Compare ten open source AI gateway options with managed services, including licenses, deployment tradeoffs and a worked ownership cost example.
