Azure had the fastest median time to first token among major providers in May 2026 at 548 ms. The median major provider took 1.24 s.

Key findings

1. 01Fastest median TTFT: Azure at 548 ms, then Nebius at 819 ms.
2. 02Highest median output speed: Vertex (Gemini) at 141 tokens per second.
3. 03Fastest median full streamed response: Azure at 980 ms.

What to do about it

• Compare providers on the same model in /rankings before switching. Provider medians mix many models.
• If p90 matters more than the median, use latency-based routing across providers for the same model.

Data

11 rows published. Rows below the floors are withheld.

Questions this answers

Which LLM API has the lowest latency?

Azure had the fastest median time to first token among major providers in May 2026 at 548 ms. The median major provider took 1.24 s.

What is the median time to first token for LLM providers?

Azure had the fastest median time to first token among major providers in May 2026 at 548 ms. The median major provider took 1.24 s.

Which provider streams the most tokens per second?

Azure had the fastest median time to first token among major providers in May 2026 at 548 ms. The median major provider took 1.24 s.

How long does a full streamed LLM response take?

Azure had the fastest median time to first token among major providers in May 2026 at 548 ms. The median major provider took 1.24 s.

Method and definitions

• Time to first token and tokens per second are measured on streamed, successful requests only, with the same formulas /rankings uses. Tokens per second counts visible output tokens after the first token, with a 250 ms floor on decode time and at least 20 output tokens.
• A provider needs at least 500 streamed successes in the month for latency to be published. Only providers with at least 0.5% of published traffic are shown.
• A row is published only when it has at least 20 requests, at least 5 distinct organizations, and no single organization above 80% of its traffic. These are the same floors /rankings uses. Rows below the floors are withheld, not estimated.
• Only shares, rates, medians and organization bands are published. Absolute request, token and spend totals stay private.

TTFT:

Time from the gateway sending the request to the first streamed token from the provider.

p90:

90% of requests were faster than this.

Total:

Time from the gateway sending the request to the last streamed token.

Per-model numbers with the same definitions are on /rankings, refreshed weekly.

Caveats

• Latency depends on the model mix. A provider serving mostly small models will look faster than one serving large reasoning models.
• Reasoning models think before the first visible token, which raises TTFT.

Revision history and other months

• October 2026first published later
• September 2026first published later
• August 2026first published later
• July 2026first published later
• June 2026first published later
• May 2026 (this page)first published 2026-11-11

Related

• Provider error ratesHow often do LLM providers return errors, and what kind?
• Prompt cache hit rate by providerWhat share of input tokens do LLM providers serve from prompt cache?
• Latency leaderboard, October 2026 (top 10 providers by volume)April 2026 note
• p50 latency year over year: October 2025 to October 2026April 2026 note
• Provider throughput density, October 2026April 2026 note
• Streaming TTFT vs total latency, April 2026April 2026 note

Cite and reuse

Free to reuse under CC BY 4.0. Credit "Requesty Data" and link to this page. Cite the month URL so the numbers you quote stay fixed.

Requesty (2026). Time to first token by provider, May 2026. Requesty Data. https://www.requesty.ai/data/provider-latency/2026-05

@misc{requesty_provider_latency_2026_05,
  author       = {{Requesty}},
  title        = {Time to first token by provider, May 2026},
  year         = {2026},
  howpublished = {\url{https://www.requesty.ai/data/provider-latency/2026-05}},
  note         = {Requesty Data}
}

period,provider,ttft_p50_ms,ttft_p90_ms,stream_total_p50_ms,tps_p50,stream_share_fraction,sample_band
2026-05,Azure,548,880,980,85.7,0.7983,100-499
2026-05,Nebius,819,1806,3492,130,0.0464,100-499
2026-05,DeepSeek,875,1674,5401,44,0.126,100-499
…