How to manage API keys for multiple LLM providers

Keep provider credentials in a gateway or vault and give each team or agent its own gateway key. As of November 2026, Requesty lets you set a monthly cap, an expiry, and model access rules for each workload, so you can expire one leaked key without replacing every other team's key.

The gateway needs protecting too. In a February 2026 scan, Wiz found that 294 of 3,074 public LiteLLM gateways accepted the example key sk-1234 or required no authentication.

Put provider credentials behind a gateway

An AI gateway is an API endpoint between your applications and model providers. It holds the provider credentials, or retrieves them from a vault, while applications authenticate to the gateway with separate keys.

Team or agent A -> gateway key A -> gateway -> OpenAI
Team or agent B -> gateway key B -> gateway -> Anthropic
                                      |
                              provider credentials

This separation gives you a smaller revocation boundary. If agent A's gateway key leaks, you can expire and replace that key without changing agent B's key. If a provider credential leaks, rotate it with the provider and update the credential stored in the gateway.

Centralizing credentials also makes the gateway worth protecting. Of the gateways in Wiz's finding, 191 had no key set. Access could expose stored provider keys and, in Wiz's tests, reach cloud IAM credentials.

If you self-host a gateway, change its example admin credentials and protect the host as a credential store. We cover the LiteLLM exposure finding in more detail in our security post.

LiteLLM has one rotation detail to check: LITELLM_SALT_KEY falls back to the master key when it is unset. Rotating the master key in that configuration can leave stored provider credentials unreadable. Before changing it, follow LiteLLM's master key rotation procedure for your existing salt-key configuration.

Where native provider limits apply

Provider controls remain useful behind a gateway. They add a second spending boundary at the provider, while gateway keys separate workloads that share a provider account.

Anthropic organizations can have up to 100 workspaces by default. OpenAI admins can also restrict which kinds of keys can be created and enforce a maximum key lifetime.

The response when a limit is reached differs too. OpenAI returns HTTP 429 for requests affected by a hard limit, while Requesty returns HTTP 412 when a key reaches its monthly cap. Handle both responses in your application so a budget stop does not look like an unexplained provider failure.

Set up a key for each workload

1. 1
   Choose where provider credentials live

   Store provider credentials in the gateway or in a vault the gateway can access. With Requesty BYOK, add provider keys in the console. An organization can store at most one key per provider.

2. 2
   Create separate gateway keys

   Create a key for each team or agent that needs its own spending and revocation boundary. Follow the service account instructions to create a key under a service account in the Requesty console. Each key can have its own limit, expiration, and logging settings. Separate keys also make it easier to attribute spend to each agent.

3. 3
   Set access and spending rules

   Set a monthly cap and expiry for each key. Use access lists and approved-model policies to restrict which models the workload can call, and add team or organization limits where you need a wider boundary. An access list attached directly to a key overrides the list on its group.

4. 4
   Configure a fallback policy

   To use your own provider keys as in the worked example, follow the BYOK instructions and create a fallback policy. Select "My own key" for each model the workload will use, apply the policy to the workload's gateway key, and use the policy as the model ID.

5. 5
   Move applications to the gateway

   Configure each application to use its gateway key and the gateway endpoint. Requesty's OpenAI-compatible base URL is https://router.requesty.ai/v1. For the BYOK policy, use the policy as the model ID. Once a workload is using the gateway, remove its direct provider credentials from the application configuration.

   Store each gateway key in the workload's secret manager or CI secret store and inject it at runtime. Keep it out of repositories and logs; see GitHub's secure-use guidance.

6. 6
   Review and expire keys

   Review unused keys, keys without spending limits, and upcoming expirations. Requesty's API Keys page can filter for dormant, never-used, and uncapped keys, and can bulk-edit expiry, limits, and access lists.

For a larger rollout, this can reduce how many people need provider credentials. In one Requesty customer rollout, provider credentials stayed in the gateway instead of being handed to 1,300 engineers, and policies cascaded from the organization to teams and individual keys.

Size the cap around the work

A dollar cap needs a workload estimate. Take a coding agent expected to run 500 tasks per month, with access limited to anthropic/claude-sonnet-5-5.

Assume each task costs $0.52, the Claude Sonnet 5.5 figure from Requesty's October 2026 coding benchmark, which ran the same agent across 129 tasks. The estimates below cover model costs only.

A $320 monthly cap would leave $60 above the first estimate. At that rate, it would cover about 615 tasks before the cap is reached ($320 divided by $0.52, rounded down).

Cost per task is the number most likely to differ. Your agent's prompts and workload won't match the benchmark, so use the estimate to set an initial cap, then adjust it against observed spend. The benchmark measured 87.6% solved for Claude Sonnet 5.5 and 92.2% for Claude Opus 5.5, so the cheaper model's cost comes with a different result.

The model allowlist affects the cap too. At the benchmark's $0.93 per task for Claude Opus 5.5, the same $320 would cover about 344 tasks. An agent allowed to switch models can consume its budget at a different rate from the one you planned for. For more on setting the boundary, see our guide to LLM spending limits per API key.

Choose where to run the gateway

The implementation determines who stores the credentials and who maintains the gateway. These options have different controls, so check the boundary before moving keys.

For LiteLLM, virtual-key budgets do not cap spending on a deployment without a database. For Portkey, review which workspaces can use each secret reference before leaving its default access in place.

When a key leaks

Expire the affected gateway key first, then issue a replacement to the workload. Review the key's usage and check whether the incident exposed a provider credential or only the gateway key.

If a provider credential was exposed, rotate it with the provider and update the gateway's stored credential. If the gateway itself was exposed, review the other credentials it could access before returning it to service.

Start with one agent that already uses more than one model. Give it a separate capped key, move it to the gateway, and compare its observed monthly spend with the estimate before applying the same setup to the rest of the team.