Previous edition: Latency leaderboard per provider, April 2026
Latency leaderboard, October 2026 (top 10 providers by volume)
Latency leaderboard, October 2026 (top 10 providers by volume)
Switch between p50, p95 and time-to-first-token. Hover any row for all three.
Median time from request to last token, successful only.Latency only counted on successful requests. TTFT only counted on streamed-and-successful requests.
Which AI providers are fastest? Among the ten busiest providers in October 2026, Typesafe had the lowest median total latency at 225ms and DeepInfra the highest at 8.0s. DeepInfra had the longest tail with a p95 of 135.6s.
Why it mattersMedian latency shows typical responsiveness, while p95 shows how bad the slow requests get. Both depend heavily on the models each provider serves, so compare providers that serve similar models.
Period
Oct 2026
Updated
November 11, 2026
ID
latency-leaderboard-october-2026
§ 01
Key findings
- 01Typesafe: p50 225ms, p95 328ms.
- 02Nebius: p50 287ms, p95 1.5s.
- 03TensorX: p50 360ms, p95 17.7s.
- 04Vertex (Gemini): p50 819ms, p95 18.4s.
§ 01b
Questions this answers
- Which AI provider has the lowest latency?
- Which provider has the worst tail latency?
- What is the typical time to first token by provider?
§ 02
Data
| Provider | p50 latency(milliseconds) | p95 latency(milliseconds) | p50 TTFT(milliseconds) |
|---|---|---|---|
| Vertex (Gemini) | 819 ms | 18.4 s | 3.54 s |
| Azure | 2.74 s | 16.5 s | 1.73 s |
| Typesafe | 225 ms | 328 ms | 224 ms |
| TensorX | 360 ms | 17.7 s | 1.32 s |
| OpenAI | 2.20 s | 15.5 s | 2.43 s |
| DeepInfra | 7.97 s | 135.6 s | 1.51 s |
| Nebius | 287 ms | 1.54 s | 693 ms |
| Bedrock | 4.39 s | 58.7 s | 2.99 s |
| Sference | 2.67 s | 48.3 s | 657 ms |
| Anthropic | 4.00 s | 31.0 s | 2.95 s |
§ 03
Cite as
APA
Click to copy
BibTeX
Click to copy
ID: latency-leaderboard-october-2026·Updated November 11, 2026·Period Oct 2026