Previous edition: Provider throughput density, April 2026
Provider throughput density, October 2026
Provider throughput density, October 2026
p50 of output tokens emitted per second of total wall-clock latency. Reads as 'how fast does this provider stream tokens once a request is in flight?'
Higher is faster. p50 of (output_tokens / total_latency_seconds).
Typesafe leads at 889 tokens per second. Parasail is slowest at 28.Successful completions only; output_tokens > 0 and total_latency > 0. Vertex split applied; direct google traffic excluded; providers with fewer than 50,000 October requests excluded. Source: Requesty production gateway.
Which providers emit tokens fastest? In October 2026 Typesafe led at a median 889 output tokens per second of total latency. Parasail was slowest at 28 tokens per second.
Why it mattersTokens per second of wall-clock time is a rough measure of how quickly a provider turns a request into output. It depends on the model mix, so a provider serving small models will look faster.
Period
Oct 2026
Updated
November 11, 2026
ID
provider-throughput-october-2026
§ 01
Key findings
- 01Typesafe: 889 tokens/sec.
- 02DeepSeek: 149 tokens/sec.
- 03Melious: 106 tokens/sec.
- 04Anthropic: 88 tokens/sec.
§ 01b
Questions this answers
- Which AI provider has the highest tokens per second?
- How fast do providers generate output tokens?
- Which providers are slowest per token?
§ 02
Data
| Provider | p50 tokens / sec | p50 ms / token(milliseconds) |
|---|---|---|
| Vertex (Gemini) | 49 | 20 ms |
| Azure | 68 | 15 ms |
| Typesafe | 889 | 1 ms |
| TensorX | 37 | 27 ms |
| OpenAI | 45 | 22 ms |
| DeepInfra | 37 | 27 ms |
| Nebius | 82 | 12 ms |
| Bedrock | 51 | 20 ms |
| Sference | 79 | 13 ms |
| Anthropic | 88 | 11 ms |
| DeepSeek | 149 | 7 ms |
| Vertex (Claude) | 84 | 12 ms |
| Novita | 31 | 33 ms |
| Melious | 106 | 10 ms |
| Parasail | 28 | 36 ms |
§ 03
Cite as
APA
Click to copy
BibTeX
Click to copy
ID: provider-throughput-october-2026·Updated November 11, 2026·Period Oct 2026