Route every request to the right model.
Fail over across providers, split traffic by share, or send each request to whichever model is fastest right now. One API, 600+ models.
Pick the models. Pick how they are chosen.
A policy is a list of models and a strategy. Nest them when one is not enough.
Failover
Tried in the order you set. A 429 or 5xx on one moves the request to the next.
Load balance
Split by the shares you set. Most traffic to the proven model, a slice to the challenger.
Latency
Fastest right now wins, scored on time to first token and generation speed. Order is ignored.
One string in the model field
Build the policy in the dashboard, then name it where the model used to go. Reorder the chain, change a share or add a model, and nothing ships.
- Nest policiesA latency policy per region, a failover across them
- Stay in regionA chain of region-pinned models never fails over out of it
- Sticky when it mattersPass a trace_id and the ordering holds for that trace
response = client.chat.completions.create( model="policy/prod-chain", messages=messages, ) # the chain, shares and fallbacks live in the policy
Four gateways. You choose.
Point your client at the EU, US or AP gateway and every request is handled there. Point it at Global and it lands on the nearest one. Requesty never guesses from the caller.
- Chosen by base URLSame key and request shape on all four
- Restrict per organizationAllow EU only, and the other three reject
- Inference is separateApprove region-pinned models to keep it in region
A policy per team, per use case, per key
Set an organization default, a stricter chain for one team, a policy of its own for a batch job. Spend limits and model restrictions cascade the same way.
| Scope | Routing policy | Models it may reach | Monthly ceiling |
|---|---|---|---|
| acme-inc | prod-chain | approved list | $20,000 |
| payments | eu-only-chain | eu-region models only | $6,000 |
| dana@acme-inc | inherits prod-chain | inherits the org list | $500 |
| sk-...3f9c | nightly-batch | two cheap models | $60 |
A scope can narrow what it inherits. It can never widen it.
for callers going direct to a single provider, same month, same upstream events.
A different chain for every agent
A review agent reaches for the strongest model you allow. A batch job runs on the cheapest one you trust. Each gets its own policy, and its own cap.
Let a policy pick the model
One string in the model field. The chain behind it is yours to change.