Requesty coding benchmark/updated 29 Oct 2026/129 private tasks/25 models
Which models ship real backend code, how fast, and at what price.
We gave every model the same coding agent and 129 private tasks from our own Go backend, then checked the work against hidden tests the model never sees.
Top 5
- 1claude-opus-5-592%
- 2claude-fable-5.189%
- 3gemini-4-argon88%
- 4claude-sonnet-5-588%
- 5gpt-6-astra88%
Leaderboard
Pick a metric
Same tasks, same agent, same 80 turn budget. Switch between score, speed, effort, tokens and cost.
Tasks solved is the headline score: a task only counts when every hidden test for it passes.
Tests passed is partial credit: the share of hidden tests passed, averaged over all tasks, so a near miss still counts.
Tasks solved, higher is better
- 1claude-opus-5-592%
- 2claude-fable-5.189%
- 3gemini-4-argon88%
- 4claude-sonnet-5-588%
- 5gpt-6-astra88%
- 6gpt-6.1-sol87%
- 7gpt-6.1-terra83%
- 8claude-haiku-5-581%
- 9kimi-k3.1-pro81%
- 10mistral-large-4open78%
- 11deepseek-v4.1-flashopen77%
- 12gpt-6-sol76%
- 13hy4-previewopen74%
- 14grok-4.774%
- 15kimi-k3open72%
- 16particle-171%
- 17muse-spark-openopen71%
- 18qwen3.8-flash70%
- 19deepseek-v4-proopen68%
- 20glm-5.3open67%
- 21minimax-m3.1-flashopen67%
- 22muse-spark-1.366%
- 23gpt-6-luna65%
- 24glm-5.3-flashopen64%
- 25step-5-previewopen57%
Speed
Smart or fast? Every model on one map
Tasks solved against median seconds per model turn. Top right is where you want to be. The dashed line is the Pareto frontier: models where nothing else is both faster and better.
Pareto frontier: no other model scores higher and is also fasterHover a dot for its numbers
Output tokens per second
- 1gemini-4-argon264
- 2particle-1232
- 3claude-haiku-5-5186
- 4minimax-m3.1-flashopen185
- 5muse-spark-1.3153
- 6deepseek-v4.1-flashopen152
- 7glm-5.3-flashopen132
- 8claude-sonnet-5-5129
- 9mistral-large-4open87
- 10glm-5.3open87
Fewest agent turns per task
- 1claude-sonnet-5-513
- 2claude-opus-5-519
- 3claude-haiku-5-520
- 4gpt-6.1-sol28
- 5gpt-6-astra30
- 6muse-spark-1.331
- 7gemini-4-argon31
- 8claude-fable-5.132
- 9gpt-6.1-terra32
- 10gpt-6-sol34
Price
What a solved task costs
List prices from the Requesty catalogue and the measured cost of each run. Cheaper per token does not always mean cheaper per solved task. The dashed line is the Pareto frontier.
Pareto frontier: no other model scores higher and is also cheaperHover a dot for its numbers
| # | Model | |||||||
|---|---|---|---|---|---|---|---|---|
| 1 | gpt-6-lunaOpenAI | 65% | $0.10 | $0.50 | $0.05 | $0.07 | 22 Sept 2026 | 20 Oct 2026 |
| 2 | qwen3.8-flashAlibaba | 70% | $0.16 | $0.47 | $0.05 | $0.08 | 26 Aug 2026 | 20 Oct 2026 |
| 3 | claude-haiku-5-5Anthropic | 81% | $0.10 | $0.50 | $0.11 | $0.14 | 7 Oct 2026 | 20 Oct 2026 |
| 4 | gpt-6.1-terraOpenAI | 83% | $1.00 | $5.00 | $0.16 | $0.19 | 26 Oct 2026 | 30 Oct 2026 |
| 5 | particle-1Particle | 71% | $0.25 | $1.00 | $0.19 | $0.27 | 29 Oct 2026 | 30 Oct 2026 |
| 6 | gpt-6.1-solOpenAI | 87% | $2.00 | $10 | $0.26 | $0.30 | 29 Sept 2026 | 20 Oct 2026 |
| 7 | glm-5.3-flashZhipuopen | 64% | $0.12 | $0.46 | $0.24 | $0.38 | 26 Aug 2026 | 20 Oct 2026 |
| 8 | deepseek-v4.1-flashDeepSeekopen | 77% | $0.15 | $0.60 | $0.31 | $0.41 | 10 Sept 2026 | 20 Oct 2026 |
| 9 | step-5-previewStepFunopen | 57% | $1.00 | $2.70 | $0.28 | $0.48 | 8 Oct 2026 | 30 Oct 2026 |
| 10 | hy4-previewTencentopen | 74% | $0.83 | $2.50 | $0.38 | $0.51 | 28 Aug 2026 | 20 Oct 2026 |
| 11 | minimax-m3.1-flashMiniMaxopen | 67% | $0.15 | $0.60 | $0.37 | $0.55 | 19 Oct 2026 | 30 Oct 2026 |
| 12 | claude-sonnet-5-5Anthropic | 88% | $2.00 | $10 | $0.52 | $0.59 | 28 Sept 2026 | 20 Oct 2026 |
| 13 | gpt-6-solOpenAI | 76% | $2.00 | $10 | $0.54 | $0.71 | 22 Sept 2026 | 20 Oct 2026 |
| 14 | muse-spark-openMetaopen | 71% | $1.25 | $4.25 | $0.61 | $0.87 | 15 Oct 2026 | 30 Oct 2026 |
| 15 | claude-opus-5-5Anthropic | 92% | $4.00 | $20 | $0.93 | $1.01 | 22 Sept 2026 | 20 Oct 2026 |
| 16 | kimi-k3.1-proMoonshot | 81% | $2.00 | $10 | $0.85 | $1.04 | 23 Oct 2026 | 30 Oct 2026 |
| 17 | muse-spark-1.3Meta | 66% | $1.25 | $4.25 | $0.99 | $1.50 | 12 Oct 2026 | 30 Oct 2026 |
| 18 | glm-5.3Zhipuopen | 67% | $1.14 | $3.42 | $1.46 | $2.16 | 18 Aug 2026 | 20 Oct 2026 |
| 19 | grok-4.7xAI | 74% | $2.00 | $6.00 | $1.71 | $2.32 | 21 Sept 2026 | 20 Oct 2026 |
| 20 | kimi-k3Moonshotopen | 72% | $3.00 | $15 | $1.72 | $2.39 | 16 Jul 2026 | 20 Oct 2026 |
| 21 | gpt-6-astraOpenAI | 88% | $10 | $50 | $2.11 | $2.40 | 4 Sept 2026 | 20 Oct 2026 |
| 22 | gemini-4-argonGoogle | 88% | $2.00 | $10 | $2.14 | $2.43 | 13 Oct 2026 | 20 Oct 2026 |
| 23 | deepseek-v4-proDeepSeekopen | 68% | $1.75 | $3.50 | $1.82 | $2.66 | 24 Apr 2026 | 20 Oct 2026 |
| 24 | mistral-large-4Mistralopen | 78% | $0.68 | $2.09 | $2.36 | $3.04 | 6 Oct 2026 | 29 Oct 2026 |
| 25 | claude-fable-5.1Anthropic | 89% | $10 | $50 | $3.48 | $3.90 | 1 Sept 2026 | 20 Oct 2026 |
Difficulty
Where models start to break
Share of tasks solved at each level. Easy and medium are nearly solved; extra hard and extreme separate the field.
| Model | easy | medium | hard | xhard | xtreme | all |
|---|---|---|---|---|---|---|
| claude-opus-5-5 | 93% | 100% | 97% | 87% | 78% | 92% |
| claude-fable-5.1 | 100% | 100% | 97% | 80% | 61% | 89% |
| gemini-4-argon | 93% | 100% | 100% | 80% | 56% | 88% |
| claude-sonnet-5-5 | 100% | 100% | 97% | 70% | 67% | 88% |
| gpt-6-astra | 100% | 100% | 97% | 73% | 61% | 88% |
| gpt-6.1-sol | 100% | 100% | 97% | 67% | 67% | 87% |
| gpt-6.1-terra | 100% | 100% | 92% | 70% | 44% | 83% |
| claude-haiku-5-5 | 100% | 100% | 87% | 63% | 56% | 81% |
| kimi-k3.1-pro | 93% | 93% | 100% | 67% | 39% | 81% |
| mistral-large-4 | 93% | 100% | 95% | 60% | 22% | 78% |
| deepseek-v4.1-flash | 93% | 100% | 97% | 57% | 17% | 77% |
| gpt-6-sol | 93% | 96% | 90% | 57% | 33% | 76% |
| hy4-preview | 93% | 100% | 87% | 53% | 28% | 74% |
| grok-4.7 | 100% | 93% | 90% | 50% | 28% | 74% |
| kimi-k3 | 86% | 89% | 92% | 53% | 22% | 72% |
| particle-1 | 86% | 96% | 92% | 43% | 22% | 71% |
| muse-spark-open | 93% | 96% | 85% | 50% | 17% | 71% |
| qwen3.8-flash | 100% | 93% | 77% | 53% | 22% | 70% |
| deepseek-v4-pro | 93% | 93% | 87% | 37% | 22% | 68% |
| glm-5.3 | 100% | 86% | 90% | 43% | 6% | 67% |
| minimax-m3.1-flash | 86% | 86% | 82% | 57% | 11% | 67% |
| muse-spark-1.3 | 86% | 93% | 90% | 33% | 11% | 66% |
| gpt-6-luna | 86% | 93% | 85% | 37% | 11% | 65% |
| glm-5.3-flash | 100% | 89% | 82% | 37% | 0% | 64% |
| step-5-preview | 86% | 93% | 67% | 23% | 17% | 57% |
Open weights
mistral-large-4 leads open weights
78% of tasks solved, ranked #10 overall, at $3.04 per solved task.
- 1mistral-large-4open78%
- 2deepseek-v4.1-flashopen77%
- 3hy4-previewopen74%
- 4kimi-k3open72%
- 5muse-spark-openopen71%
- 6deepseek-v4-proopen68%
- 7glm-5.3open67%
- 8minimax-m3.1-flashopen67%
- 9glm-5.3-flashopen64%
- 10step-5-previewopen57%
All data
Everything we measured
Click a column to sort. Filter by open weights. Tasks solved needs every hidden test to pass; tests passed is partial credit. Averages per task unless noted.
| # | Model | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | claude-opus-5-5Anthropic | 92% | 96% | 8.5s | 76 | 19 | 9.5m | 1.5M | 20k | 100% | $0.93 |
| 2 | claude-fable-5.1Anthropic | 89% | 95% | 8.6s | 68 | 32 | 12.4m | 2.8M | 40k | 100% | $3.48 |
| 3 | gemini-4-argonGoogle | 88% | 95% | 5.6s | 264 | 31 | 5.5m | 5.2M | 58k | 87% | $2.14 |
| 4 | claude-sonnet-5-5Anthropic | 88% | 97% | 4.5s | 129 | 13 | 9.5m | 0.9M | 17k | 99% | $0.52 |
| 5 | gpt-6-astraOpenAI | 88% | 97% | 6.1s | 26 | 30 | 11.7m | 1.4M | 6k | 100% | $2.11 |
| 6 | gpt-6.1-solOpenAI | 87% | 97% | 5.4s | 29 | 28 | 11.5m | 1.3M | 6k | 100% | $0.26 |
| 7 | gpt-6.1-terraOpenAI | 83% | 96% | 4.7s | 36 | 32 | 6.3m | 1.5M | 7k | 100% | $0.16 |
| 8 | claude-haiku-5-5Anthropic | 81% | 94% | 4.7s | 186 | 20 | 5.4m | 1.7M | 36k | 100% | $0.11 |
| 9 | kimi-k3.1-proMoonshot | 81% | 92% | 10.3s | 25 | 37 | 11.8m | 2.9M | 13k | 96% | $0.85 |
| 10 | mistral-large-4Mistralopen | 78% | 89% | 3.1s | 87 | 47 | 10.6m | 3.4M | 30k | 0% | $2.36 |
| 11 | deepseek-v4.1-flashDeepSeekopen | 77% | 85% | 2.3s | 152 | 45 | 16.5m | 3.6M | 116k | 97% | $0.31 |
| 12 | gpt-6-solOpenAI | 76% | 94% | 3.2s | 48 | 34 | 10.7m | 1.6M | 6k | 100% | $0.54 |
| 13 | hy4-previewTencentopen | 74% | 83% | 7.0s | 29 | 60 | 45.3m | 4.2M | 52k | 97% | $0.38 |
| 14 | grok-4.7xAI | 74% | 88% | 3.6s | 59 | 35 | 10.1m | 2.5M | 13k | 86% | $1.71 |
| 15 | kimi-k3Moonshotopen | 72% | 89% | 6.2s | 39 | 42 | 11.8m | 2.8M | 16k | 96% | $1.72 |
| 16 | particle-1Particle | 71% | 87% | 2.0s | 232 | 40 | 5.6m | 2.6M | 39k | 87% | $0.19 |
| 17 | muse-spark-openMetaopen | 71% | 88% | 3.8s | 64 | 34 | 4.4m | 3.1M | 13k | 95% | $0.61 |
| 18 | qwen3.8-flashAlibaba | 70% | 84% | 5.7s | 49 | 39 | 11.9m | 2.4M | 19k | 97% | $0.05 |
| 19 | deepseek-v4-proDeepSeekopen | 68% | 84% | 4.9s | 61 | 45 | 32.2m | 3.6M | 71k | 94% | $1.82 |
| 20 | glm-5.3Zhipuopen | 67% | 78% | 2.4s | 87 | 50 | 22.7m | 3.4M | 95k | 96% | $1.46 |
| 21 | minimax-m3.1-flashMiniMaxopen | 67% | 83% | 4.2s | 185 | 66 | 20.4m | 4.4M | 196k | 98% | $0.37 |
| 22 | muse-spark-1.3Meta | 66% | 79% | 3.4s | 153 | 31 | 11.2m | 2.3M | 91k | 85% | $0.99 |
| 23 | gpt-6-lunaOpenAI | 65% | 84% | 4.1s | 70 | 47 | 11.6m | 2.6M | 19k | 100% | $0.05 |
| 24 | glm-5.3-flashZhipuopen | 64% | 73% | 1.0s | 132 | 53 | 11.4m | 3.3M | 48k | 97% | $0.24 |
| 25 | step-5-previewStepFunopen | 57% | 74% | 5.1s | 12 | 42 | 17.9m | 2.2M | 11k | 94% | $0.28 |
Method
How the benchmark works
The tasks
129 real changes from our own Go backend: 14 easy, 28 medium, 39 hard, 30 extra hard and 18 extreme. They are private, so no model has seen them in training.
The harness
Every model gets the same coding agent, the same tools (bash, read, edit, write), the same sandboxed repo and a budget of 80 turns. All traffic goes through Requesty, so every model is called the same way.
The scoring
A task counts as solved only when the hidden tests written for it pass and the existing tests still pass. The model never sees the hidden tests. Runs where the model read the answer are flagged and checked.
- Speed is measured end to end through Requesty from a single region, so it includes provider queueing and network time.
- Some providers do not count hidden reasoning in output tokens, so throughput for those models (notably the GPT family) reads lower than the work done.
- Prices are list prices per million tokens on the cheapest Requesty provider for that model. Cost per task is what each run cost us, including retries. Benchmarked is the date the run finished; for runs before October 20 it is the date the results were published.
- Updated 2026-10-29. Only models with all 129 tasks completed are shown. Open weights follows the Requesty model catalogue.