Time to first token by provider · September 2026
Sference had the fastest median time to first token among major providers in September 2026 at 267 ms. The median major provider took 1.37 s.
Median and p90 time to first token, median total response time, and median output tokens per second, for streamed successful requests per provider.
Data through September 30, 2026Updated November 11, 2026
This is the permanent snapshot for September 2026. See the latest month (October 2026)
Median time to first token, September 2026
Source: Requesty production gateway- Sference267 ms
- Azure796 ms
- Nebius983 ms
- TensorX1.02 s
- DeepSeek1.04 s
- DeepInfra1.14 s
- OpenAI1.16 s
- Vertex (Claude)1.24 s
- Fireworks1.37 s
- Mistral1.46 s
- Anthropic1.52 s
- xAI1.53 s
- Bedrock1.70 s
- Runware2.00 s
- Vertex (Gemini)2.06 s
Key findings
Median time to first token by month
What changed since August 2026
- Novita1.40 s → 2.79 s (+1.39 s)
- OpenAI547 ms → 1.16 s (+616 ms)
- Nebius542 ms → 983 ms (+441 ms)
- TensorX1.35 s → 1.02 s (-337 ms)
- Sference55 ms → 267 ms (+212 ms)
What to do about it
- Compare providers on the same model in /rankings before switching. Provider medians mix many models.
- If p90 matters more than the median, use latency-based routing across providers for the same model.
Latency routing sends each request to the provider that is currently fastest for that model.
Read about latency routingData
| Provider | Median TTFT | p90 TTFT | Median total | Median tokens/s | Streamed share | Organizations (band) |
|---|---|---|---|---|---|---|
| Sference | 267 ms | 1.93 s | 2.12 s | 149.600 | 60.76% | 500+ |
| Azure | 796 ms | 4.45 s | 2.68 s | 58.100 | 36.38% | 500+ |
| Nebius | 983 ms | 3.40 s | 2.98 s | 82.200 | 6.54% | 500+ |
| TensorX | 1.02 s | 5.81 s | 3.32 s | 66.200 | 20.54% | 500+ |
| DeepSeek | 1.04 s | 2.19 s | 4.41 s | 96.800 | 24.05% | 500+ |
| DeepInfra | 1.14 s | 3.54 s | 3.29 s | 73.100 | 13.31% | 500+ |
| OpenAI | 1.16 s | 6.74 s | 6.11 s | 71.700 | 18.43% | 500+ |
| Vertex (Claude) | 1.24 s | 2.98 s | 5.54 s | 78.200 | 57.72% | 500+ |
| Fireworks | 1.37 s | 3.88 s | 4.83 s | 65.800 | 29.28% | 500+ |
| Mistral | 1.46 s | 9.60 s | 2.67 s | 137.400 | 15.68% | 500+ |
| Anthropic | 1.52 s | 3.38 s | 6.22 s | 73.800 | 18.20% | 500+ |
| xAI | 1.53 s | 4.32 s | 5.10 s | 40.900 | 16.09% | 100-499 |
| Bedrock | 1.70 s | 3.42 s | 6.00 s | 82.600 | 30.28% | 500+ |
| Runware | 2.00 s | 4.72 s | 4.11 s | 129.400 | 44.88% | 500+ |
| Vertex (Gemini) | 2.06 s | 8.24 s | 3.16 s | 169.000 | 5.40% | 500+ |
| NVIDIA | 2.22 s | 19.2 s | 10.3 s | 58.900 | 62.02% | 500+ |
| Novita | 2.79 s | 7.52 s | 6.89 s | 83.400 | 22.63% | 500+ |
17 rows published. Rows below the floors are withheld.
Questions this answers
- Which LLM API has the lowest latency?
- Sference had the fastest median time to first token among major providers in September 2026 at 267 ms. The median major provider took 1.37 s.
- What is the median time to first token for LLM providers?
- Sference had the fastest median time to first token among major providers in September 2026 at 267 ms. The median major provider took 1.37 s.
- Which provider streams the most tokens per second?
- Sference had the fastest median time to first token among major providers in September 2026 at 267 ms. The median major provider took 1.37 s.
- How long does a full streamed LLM response take?
- Sference had the fastest median time to first token among major providers in September 2026 at 267 ms. The median major provider took 1.37 s.
Method and definitions
- Time to first token and tokens per second are measured on streamed, successful requests only, with the same formulas /rankings uses. Tokens per second counts visible output tokens after the first token, with a 250 ms floor on decode time and at least 20 output tokens.
- A provider needs at least 500 streamed successes in the month for latency to be published. Only providers with at least 0.5% of published traffic are shown.
- A row is published only when it has at least 20 requests, at least 5 distinct organizations, and no single organization above 80% of its traffic. These are the same floors /rankings uses. Rows below the floors are withheld, not estimated.
- Only shares, rates, medians and organization bands are published. Absolute request, token and spend totals stay private.
- TTFT:
- Time from the gateway sending the request to the first streamed token from the provider.
- p90:
- 90% of requests were faster than this.
- Total:
- Time from the gateway sending the request to the last streamed token.
Per-model numbers with the same definitions are on /rankings, refreshed weekly.
Caveats
- Latency depends on the model mix. A provider serving mostly small models will look faster than one serving large reasoning models.
- Reasoning models think before the first visible token, which raises TTFT.
Revision history and other months
- October 2026first published later
- September 2026 (this page)first published 2026-11-11
- August 2026first published 2026-11-11
- July 2026first published 2026-11-11
- June 2026first published 2026-11-11
- May 2026first published 2026-11-11
Cite and reuse
Free to reuse under CC BY 4.0. Credit "Requesty Data" and link to this page. Cite the month URL so the numbers you quote stay fixed.
APA
Click to copy
BibTeX
Click to copy
CSV
Click to copy