Sference had the fastest median time to first token among major providers in August 2026 at 55 ms. The median major provider took 1.33 s.

Key findings

1. 01Fastest median TTFT: Sference at 55 ms, then Alibaba at 200 ms.
2. 02Highest median output speed: Vertex (Gemini) at 192 tokens per second.
3. 03Fastest median full streamed response: Alibaba at 1.07 s.
4. 04Novita moved the most since July 2026: median TTFT went from 3.71 s to 1.40 s (-2.31 s).

Median time to first token by month

What changed since July 2026

• Novita3.71 s → 1.40 s (-2.31 s)
• Alibaba1.78 s → 200 ms (-1.58 s)
• xAI970 ms → 1.47 s (+500 ms)
• OpenAI923 ms → 547 ms (-376 ms)
• Anthropic1.96 s → 1.60 s (-362 ms)

What to do about it

• Compare providers on the same model in /rankings before switching. Provider medians mix many models.
• If p90 matters more than the median, use latency-based routing across providers for the same model.

Data

17 rows published. Rows below the floors are withheld.

Questions this answers

Which LLM API has the lowest latency?

Sference had the fastest median time to first token among major providers in August 2026 at 55 ms. The median major provider took 1.33 s.

What is the median time to first token for LLM providers?

Sference had the fastest median time to first token among major providers in August 2026 at 55 ms. The median major provider took 1.33 s.

Which provider streams the most tokens per second?

Sference had the fastest median time to first token among major providers in August 2026 at 55 ms. The median major provider took 1.33 s.

How long does a full streamed LLM response take?

Sference had the fastest median time to first token among major providers in August 2026 at 55 ms. The median major provider took 1.33 s.

Method and definitions

• Time to first token and tokens per second are measured on streamed, successful requests only, with the same formulas /rankings uses. Tokens per second counts visible output tokens after the first token, with a 250 ms floor on decode time and at least 20 output tokens.
• A provider needs at least 500 streamed successes in the month for latency to be published. Only providers with at least 0.5% of published traffic are shown.
• A row is published only when it has at least 20 requests, at least 5 distinct organizations, and no single organization above 80% of its traffic. These are the same floors /rankings uses. Rows below the floors are withheld, not estimated.
• Only shares, rates, medians and organization bands are published. Absolute request, token and spend totals stay private.

TTFT:

Time from the gateway sending the request to the first streamed token from the provider.

p90:

90% of requests were faster than this.

Total:

Time from the gateway sending the request to the last streamed token.

Per-model numbers with the same definitions are on /rankings, refreshed weekly.

Caveats

• Latency depends on the model mix. A provider serving mostly small models will look faster than one serving large reasoning models.
• Reasoning models think before the first visible token, which raises TTFT.

Revision history and other months

• October 2026first published later
• September 2026first published later
• August 2026 (this page)first published 2026-11-11
• July 2026first published 2026-11-11
• June 2026first published 2026-11-11
• May 2026first published 2026-11-11

Related

• Provider error ratesHow often do LLM providers return errors, and what kind?
• Prompt cache hit rate by providerWhat share of input tokens do LLM providers serve from prompt cache?
• Latency leaderboard, October 2026 (top 10 providers by volume)April 2026 note
• p50 latency year over year: October 2025 to October 2026April 2026 note
• Provider throughput density, October 2026April 2026 note
• Streaming TTFT vs total latency, April 2026April 2026 note

Cite and reuse

Free to reuse under CC BY 4.0. Credit "Requesty Data" and link to this page. Cite the month URL so the numbers you quote stay fixed.

Requesty (2026). Time to first token by provider, August 2026. Requesty Data. https://www.requesty.ai/data/provider-latency/2026-08

@misc{requesty_provider_latency_2026_08,
  author       = {{Requesty}},
  title        = {Time to first token by provider, August 2026},
  year         = {2026},
  howpublished = {\url{https://www.requesty.ai/data/provider-latency/2026-08}},
  note         = {Requesty Data}
}

period,provider,ttft_p50_ms,ttft_p90_ms,stream_total_p50_ms,tps_p50,stream_share_fraction,sample_band
2026-08,Sference,55,142,1796,152.7,0.7424,500+
2026-08,Alibaba,200,4537,1071,110.2,0.3376,100-499
2026-08,Nebius,542,3197,2222,130.7,0.2346,500+
…