| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | Llama 3.1-405B Open source Meta AI | 89.2% | — | 23 Jul 2024 | — | Source ↗ | |
| 2 | Falcon-180B Open source Technology Innovation Institute | 87.1% | — | 6 Sep 2023 | — | Source ↗ | |
| 3 | DeepSeek-V2 (MoE-236B, May 2024) Open source DeepSeek | 86.3% | — | 7 May 2024 | — | Source ↗ | |
| 4 | DeepSeek V3 Open source DeepSeek | 85.2% | — | 26 Dec 2024 | — | Source ↗ | |
| 5 | DeepSeek-Coder-V2 236B Open source DeepSeek | 83.7% | — | 17 Jun 2024 | — | Source ↗ | |
| 6 | Llama 3-70B Open source Meta AI | 83.5% | — | 18 Apr 2024 | — | Source ↗ | |
| 7 | Qwen2.5-72B Open source Alibaba | 82.3% | — | 19 Sep 2024 | — | Source ↗ | |
| 8 | phi-3-small 7.4B Open source Microsoft | 81.5% | — | 23 Apr 2024 | — | Source ↗ | |
| 9 | phi-3-medium 14B Open source Microsoft | 81.5% | — | 23 Apr 2024 | — | Source ↗ | |
| 10 | Qwen2.5-Coder-32B Open source Alibaba | 80.8% | — | 18 Sep 2024 | — | Source ↗ | |
| 11 | Llama 2-70B Open source Meta AI | 80.2% | — | 18 Jul 2023 | — | Source ↗ | |
| 12 | Gemma 7B Open source Google DeepMind | 79.0% | — | 21 Feb 2024 | — | Source ↗ | |
| 13 | Falcon 2 11B Open source Technology Innovation Institute | 78.3% | — | 9 May 2024 | — | Source ↗ | |
| 14 | Mixtral 8x7B Open source Mistral AI | 77.2% | — | 11 Dec 2023 | — | Source ↗ | |
| 15 | LLaMA-65B Open source Meta AI | 77.0% | — | 24 Feb 2023 | — | Source ↗ | |
| 16 | Falcon-40B Open source Technology Innovation Institute | 76.9% | — | 25 May 2023 | — | Source ↗ | |
| 17 | LLaMA-33B Open source Meta AI | 76.0% | — | 24 Feb 2023 | — | Source ↗ | |
| 18 | Llama 3-8B Open source Meta AI | 75.7% | — | 18 Apr 2024 | — | Source ↗ | |
| 19 | Mistral 7B v0.1 Open source Mistral AI | 75.3% | — | 27 Sep 2023 | — | Source ↗ | |
| 20 | Phi-1.5 Open source Microsoft | 73.4% | 5 | 11 Sep 2023 | — | Source ↗ | |
| 21 | LLaMA-13B Open source Meta AI | 73.0% | — | 24 Feb 2023 | — | Source ↗ | |
| 22 | Qwen2.5-Coder (7B) Open source Alibaba | 72.9% | — | 18 Sep 2024 | — | Source ↗ | |
| 23 | Llama 2-13B Open source Meta AI | 72.8% | — | 18 Jul 2023 | — | Source ↗ | |
| 24 | Yi 6B Open source 01.AI | 71.3% | — | 22 Nov 2023 | — | Source ↗ | |
| 25 | MPT-30B Open source MosaicML | 71.0% | — | 22 Jun 2023 | — | Source ↗ | |
| 26 | phi-3-mini 3.8B Open source Microsoft | 70.8% | — | 23 Apr 2024 | — | Source ↗ | |
| 27 | LLaMA-7B Open source Meta AI | 70.1% | — | 24 Feb 2023 | — | Source ↗ | |
| 28 | Llama 2-7B Open source Meta AI | 69.2% | — | 18 Jul 2023 | — | Source ↗ | |
| 29 | MPT-7B Open source MosaicML | 68.6% | — | 5 May 2023 | — | Source ↗ | |
| 30 | Falcon-7B Open source Technology Innovation Institute | 67.2% | — | 24 Apr 2023 | — | Source ↗ | |
| 31 | Gemma 2B Open source Google DeepMind | 65.4% | — | 21 Feb 2024 | — | Source ↗ | |
| 32 | XGen-7B Open source Salesforce | 64.9% | — | 27 Jun 2023 | — | Source ↗ | |
| 33 | StarCoder 2 15B Open source Hugging Face,ServiceNow,NVIDIA,BigCode | 64.3% | — | 20 Feb 2024 | — | Source ↗ | |
| 34 | DeepSeek Coder 33B Open source DeepSeek,Peking University | 62.0% | — | 2 Nov 2023 | — | Source ↗ | |
| 35 | Dolly 2.0-12b Open source Databricks | 61.8% | — | 11 Apr 2023 | — | Source ↗ | |
| 36 | Cerebras-GPT-13B Open source Cerebras Systems | 60.8% | — | 20 Mar 2023 | — | Source ↗ | |
| 37 | Qwen2.5-Coder (1.5B) Open source Alibaba | 60.7% | — | 18 Sep 2024 | — | Source ↗ | |
| 38 | DeepSeek Coder 6.7B Open source DeepSeek,Peking University | 57.6% | — | 2 Nov 2023 | — | Source ↗ | |
| 39 | StarCoder 2 3B Open source Hugging Face,ServiceNow,NVIDIA,BigCode | 57.1% | — | 22 Feb 2024 | — | Source ↗ | |
| 40 | StarCoder 2 7B Open source Hugging Face,ServiceNow,NVIDIA,BigCode | 57.1% | — | 20 Feb 2024 | — | Source ↗ | |
| 41 | Phi-2 Open source Microsoft | 54.7% | — | 12 Dec 2023 | — | Source ↗ | |
| 42 | DeepSeek Coder 1.3B Open source DeepSeek,Peking University | 53.3% | — | 2 Nov 2023 | — | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.