games benchmark · included in ECI
Balrog
Balrog benchmark (score column: Average progress).
Results
14
Random baseline
0.0%
Score ceiling
100%
Released
20 Nov 2024
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | R1 Open source DeepSeek | 34.9% | — | 20 Jan 2025 | — | Source ↗ | |
| 2 | Reka Flash 3 Open source rekaai | 29.2% | — | 12 Mar 2025 | — | Source ↗ | |
| 3 | Llama 3.1-70B Open source Meta AI | 27.9% | — | 23 Jul 2024 | — | Source ↗ | |
| 4 | Llama 3.2 90B Open source Meta AI | 27.3% | — | 24 Sep 2024 | — | Source ↗ | |
| 5 | Llama 3.3 70B Open source Meta AI | 23.0% | — | 6 Dec 2024 | — | Source ↗ | |
| 6 | DeepSeek-R1-Distill-Qwen-32B Open source DeepSeek | 19.5% | — | 20 Jan 2025 | — | Source ↗ | |
| 7 | Mistral Nemo Open source Mistral | 17.6% | — | 18 Jul 2024 | — | Source ↗ | |
| 8 | Llama 3.2 11B Open source Meta AI | 16.8% | — | 24 Sep 2024 | — | Source ↗ | |
| 9 | Qwen2.5-72B Open source Alibaba | 16.2% | — | 19 Sep 2024 | — | Source ↗ | |
| 10 | Llama 3.1-8B Open source Meta AI | 15.1% | — | 23 Jul 2024 | — | Source ↗ | |
| 11 | Phi 4 Open source Microsoft | 11.6% | — | 12 Dec 2024 | — | Source ↗ | |
| 12 | Llama 3.2 3B Open source Meta AI | 10.1% | — | 24 Sep 2024 | — | Source ↗ | |
| 13 | Qwen2.5-7B Open source Alibaba | 7.8% | — | 19 Sep 2024 | — | Source ↗ | |
| 14 | Llama 3.2 1B Open source Meta AI | 6.6% | — | 24 Sep 2024 | — | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.