Watch our router pick the fastest provider, in real time.
Requesty routes every latency-optimized request with Thompson Sampling: one draw from each provider's live latency distribution, lowest draw wins. Sending everything to one provider is the naive strategy this replaces; the traffic split you see is the router exploiting the fastest option while auditing the rest. Refreshed every minute from production traffic. This applies to requests using our latency routing policy: enable it on your API key to get routed like this.
Live traffic share by model family46 matchups
Where our router sends each model's traffic right now. The share that doesn't go to the leader is deliberate: continuous probing of every provider, so degradations are caught within minutes.
gpt-4.1-nano
1azure @francecentral80.5%481ms713ms–$0.112openai17.0%596ms901ms–$0.013azure @swedencentral2.4%1.19s2.77s7%$0.144azure0.1%1.02s1.72s–$0.14gemini-2.5-flash-lite
1vertex @europe-west1↑13.6% err20.1%1.29s8.12s4%$0.192vertex @europe-west81.8% err19.0%1.19s5.52s2%$0.133vertex @europe-central2⚡1.5% err15.7%1.30s6.58s–$0.134vertex @europe-west48.1% err15.5%1.28s6.94s5%$0.175vertex @us-east1↑13.1%1.35s3.65s10%$0.126vertex @europe-north112.5%1.34s5.91s–$0.137vertex↑15.6% err4.0%1.85s4.84s–$0.128vertex @us-west10.1%2.41s8.99s23%$0.129google↑21.9% err0.0%18.5s35.4s–$0.3610vertex @us-central10.0%4.24s11.8s54%$0.08deepseek-v4-flash
1sference78.1%1.41s8.14s89%$0.082fireworks↑13.2%3.58s11.6s15%$0.173deepseek3.8%4.75s15.0s50%$0.094tensorx↑3.2%7.82s31.7s30%$0.125novita1.7%5.38s15.2s92%$0.14gemini-3.1-flash-lite
1google38.0%1.84s2.88s16%$0.292vertex @us29.7%1.82s3.27s29%$0.283vertex @eu↑26.2%2.06s4.20s7%$0.344vertex2.7% err6.1%2.39s4.19s2%$0.44claude-haiku-4-5
1vertex83.4%860ms1.21s–$1.732vertex @europe-west1⚡1.1% err12.5%1.76s6.72s32%$1.193bedrock @eu-central-12.0%1.89s4.31s30%$1.024bedrock @eu-west-11.1%1.79s4.12s2%$1.395bedrock @eu-north-10.7%2.12s4.83s2%$1.516anthropic↑0.2%12.6s31.2s16%$1.697bedrock @eu-west-30.1%2.81s6.25s58%$0.76gpt-4.1-mini
1openai69.1%1.01s1.80s4%$0.112azure @francecentral↑29.5%1.27s2.39s7%$0.393azure @westus30.8%3.48s15.3s–$0.774azure @eastus20.5%8.89s63.5s–$0.86gemini-2.5-flash
1coding22.6%2.99s6.00s–$1.042coding @europe-west819.4%2.60s4.66s–$1.173coding @europe-north117.8%2.96s5.89s–$1.224vertex @us-east117.3%4.16s4.19s–$1.825coding @europe-central210.2%4.07s12.9s–$1.496coding @europe-west410.1%3.23s9.89s–$1.177coding @europe-west11.6%4.01s6.12s–$1.358vertex↑7.3% err0.6%6.55s18.2s2%$1.369vertex @europe-west10.2%10.4s32.6s12%$1.4110google↑0.0%14.8s38.9s–$1.6211vertex @europe-west414.6% err0.0%13.3s60.7s5%$1.3912vertex @europe-north18.1% err0.0%15.8s27.2s2%$1.4713vertex @europe-west818.2% err0.0%18.7s31.7s4%$1.3814vertex @us-south10.0%11.0s22.9s–$1.77gpt-5.6-luna
1azure @swedencentral0.5% err86.5%2.14s5.43s22%$1.212openai⚡10.5%6.43s26.6s71%$0.463azure @eastus2↑3.0%7.67s22.5s90%$0.22gpt-4.1
1azure @francecentral56.2%1.05s1.77s63%$1.132azure @swedencentral30.0%1.34s5.09s38%$1.633openai13.9%1.52s3.57s14%$2.294azure0.0%18.8s33.6s20%$3.09gpt-5-mini
1azure @francecentral↑83.2%2.28s5.21s5%$0.572azure @swedencentral↑15.2%4.19s11.0s26%$0.973azure1.5%6.81s16.5s15%$0.524azure @uksouth0.0%10.7s22.8s5%$0.475openai↑0.0%10.3s16.5s4%$1.05glm-5.2
1sference⚡75.4%1.85s21.3s92%$0.382tensorx13.4%5.63s32.0s87%$0.583fireworks7.0%7.11s50.2s87%$0.364zai2.9%8.33s69.7s50%$1.935inceptron⚡0.6%18.3s106.3s66%$0.886nebius0.6%23.8s118.9s–$2.24claude-opus-4-8
1bedrock @eu-west-3↑38.5%3.45s18.1s41%$3.922anthropic↑21.2%4.68s22.2s30%$5.723vertex @eu13.9%5.96s40.7s84%$1.844vertex12.3%5.23s25.9s73%$4.185bedrock @eu-north-15.5%7.89s46.9s80%$1.966bedrock @eu-central-1↑4.4%7.68s33.4s81%$2.007bedrock @eu-west-13.3%9.68s47.6s60%$3.188bedrock1.0%9.93s34.3s34%$4.66claude-opus-4-5
1vertex85.7%3.66s7.04s7%$6.542bedrock11.1%6.99s17.8s12%$6.023anthropic3.2%7.65s14.8s6%$6.03claude-sonnet-4-6
1vertex @europe-west118.4%4.71s35.1s56%$2.102bedrock @eu-west-116.2%4.57s29.1s42%$2.943bedrock @eu-central-114.0%5.25s35.3s64%$2.054bedrock @us-east-1↑13.9%6.98s48.8s–$5.915bedrock @eu-west-312.5%4.99s46.1s84%$1.146vertex11.0%4.37s8.07s17%$3.887anthropic⚡7.4%7.59s76.0s56%$2.518bedrock↑3.4%6.36s31.1s–$5.839bedrock @eu-north-13.1%6.62s25.2s87%$0.94gpt-5.4
1azure @swedencentral↑34.1%2.54s20.1s38%$3.052azure @francecentral29.4%2.54s17.9s60%$1.633azure @eastus2⚡26.1%2.53s14.5s15%$3.094openai⚡1.8% err10.4%3.65s22.7s49%$2.08gemini-3.5-flash
1vertex↑48.9%4.75s27.5s36%$2.002vertex @eu5.1% err29.4%5.52s21.9s20%$2.453vertex @us21.8%6.99s29.1s21%$4.40claude-sonnet-5
1vertex @eu40.3%5.34s37.9s78%$0.852bedrock @eu-central-115.2%7.87s47.3s35%$2.693bedrock⚡13.6%6.87s22.1s58%$1.144anthropic13.6%8.00s43.2s66%$1.085vertex6.3%12.5s41.3s86%$0.636bedrock @eu-west-26.1%11.7s35.9s79%$0.827bedrock @eu-north-15.0%10.2s34.6s65%$1.14gpt-5.4-mini
1azure83.8%721ms1.33s1%$1.072openai15.2%1.27s5.33s18%$0.803azure @eastus21.0%1.19s1.65s54%$0.43minimax-m3
1fireworks67.4%3.04s8.21s11%$0.332minimaxi28.5%4.43s40.0s59%$0.313tensorx4.1%10.4s51.1s90%$0.19gpt-5.1
1azure @francecentral↑51.6%2.38s9.74s2%$2.602azure @swedencentral↑47.0%2.55s10.8s9%$2.013openai1.4%5.57s8.74s80%$1.10deepseek-v4-pro
1tensorx85.3%2.65s10.4s90%$0.602fireworks↑10.9%9.36s81.1s23%$2.203deepseek↑3.2%15.3s83.0s81%$0.164nebius0.6%16.0s49.5s–$1.76gpt-5-nano
1openai↑46.7%6.53s21.4s9%$0.292azure45.9%6.44s13.9s–$0.143azure @francecentral↑7.4%14.4s49.9s29%$0.234azure @swedencentral0.0%31.3s50.4s19%$0.31gpt-5.6-terra
1openai↑56.5%3.75s13.6s87%$0.802azure @swedencentral1% err26.4%5.78s36.9s72%$1.203azure @eastus217.1%5.52s13.8s67%$1.17claude-opus-4-7
1vertex29.6%3.22s10.8s53%$4.082vertex @eu25.1%3.71s18.0s88%$1.463bedrock @eu-north-119.4%3.65s11.4s65%$3.694bedrock9.6%10.7s113.7s35%$6.705bedrock @eu-central-18.4%5.07s22.5s88%$1.666bedrock @eu-west-1↑4.0%10.7s45.6s39%$5.147anthropic3.9%6.67s26.3s91%$1.11gpt-5.6-sol
1azure @swedencentral1.3% err50.0%6.90s37.4s82%$1.982openai42.6%7.51s24.1s92%$1.113azure @eastus27.4%15.4s61.0s91%$1.11claude-opus-4-6
1vertex @europe-west153.1%6.70s28.9s72%$3.262anthropic↑23.2%12.4s139.7s63%$3.903bedrock22.3%12.9s103.9s–$23.34vertex1.5%30.1s142.6s22%$5.77claude-sonnet-4-5
1vertex26.0%4.24s16.8s26%$3.732anthropic18.3%4.88s19.9s39%$2.923bedrock @eu-central-116.4%3.89s5.82s4%$3.874bedrock @eu-west-314.7%4.09s6.51s–$3.885bedrock @eu-north-111.3%4.08s5.87s–$3.826vertex @europe-west1⚡10.6%8.95s60.8s29%$3.237bedrock @eu-west-12.8%9.21s39.9s15%$7.87gemini-2.5-pro
1coding56.5%7.94s12.6s–$5.032vertex @europe-west1↑21.7%10.2s23.2s3%$4.123coding @europe-central212.2%15.5s35.5s3%$2.704google6.9%10.8s17.4s–$6.035vertex2.7%14.0s25.4s2%$6.666vertex @europe-central20.0%27.4s57.7s14%$2.43claude-fable-5
1anthropic48.0%10.0s50.2s50%$10.22vertex35.5%14.4s327.9s38%$12.33vertex @eu16.5%24.7s113.7s44%$9.71kimi-k2.7-code
1moonshot54.0%2.69s21.3s95%$0.692tensorx33.6%3.55s9.76s67%$0.833fireworks11.9%5.39s21.4s67%$0.854parasail0.5%24.6s108.6s59%$0.51All models with live data85 models
single provider, no routing contestHow the router decides
Measure every request
Each production request records time to first token and total latency per provider, model, region, stream mode, and input size. Stats are outlier-trimmed and recency-weighted with a 10 minute half-life, so a provider that degrades shows up within minutes.
Sample, don't average
For each request, the router draws one latency sample from every candidate's live distribution (Thompson Sampling) and routes to the lowest draw. Providers with little data get wider distributions, so new candidates keep getting explored without dedicated canary traffic.
Penalize unreliability
Candidates returning capacity errors (429s and 5xxs) over the last 15 minutes have their samples inflated by a penalty factor, up to 10x. An unreliable provider loses routing share immediately and wins it back as soon as it recovers.
Reading the board
Win share is the percentage of simulated routing decisions each candidate wins against the others in its group, using the same sampler the router runs in production. Latency is the geometric mean of total response time over the last hour. The winner is not always the lowest average: a candidate with volatile latency or recent capacity errors loses share to a steadier one. Groups split by streaming mode and input size because provider rankings flip between them.
/public/v1/routing-leaderboard