coding benchmark · included in ECI
CadEval
CadEval benchmark (score column: Overall pass (%)).
Results
14
Random baseline
0.0%
Score ceiling
100%
Released
22 Apr 2025
Score by model release date
Best score by organisation
Leaderboard
| # | Model | Score | Relative | Setting | Released | Run | Evidence |
|---|---|---|---|---|---|---|---|
| 1 | o3 (medium) OpenAI | 74.0% | medium | 16 Apr 2025 | — | Source ↗ | |
| 2 | Gemini 2.5 Pro Preview (Mar 2025) Google DeepMind | 64.0% | — | 31 Mar 2025 | — | Source ↗ | |
| 3 | o4-mini (medium) OpenAI | 62.0% | medium | 16 Apr 2025 | — | Source ↗ | |
| 4 | o1 (medium) OpenAI | 56.0% | medium | 17 Dec 2024 | — | Source ↗ | |
| 5 | Claude 3.7 Sonnet Anthropic | 54.0% | — | 24 Feb 2025 | — | Source ↗ | |
| 6 | o3-mini (medium) OpenAI | 54.0% | medium | 31 Jan 2025 | — | Source ↗ | |
| 7 | Claude 3.5 Sonnet (Oct 2024) Anthropic | 48.0% | — | 22 Oct 2024 | — | Source ↗ | |
| 8 | GPT-4.1 OpenAI | 42.0% | — | 14 Apr 2025 | — | Source ↗ | |
| 9 | Gemini 1.5 Pro (Sept 2024) Google DeepMind | 34.0% | — | 24 Sep 2024 | — | Source ↗ | |
| 10 | Claude 3.5 Haiku (Oct 2024) Anthropic | 32.0% | — | 22 Oct 2024 | — | Source ↗ | |
| 11 | Gemini 2.0 Flash (Feb 2025) Google DeepMind,Google | 30.0% | — | 5 Feb 2025 | — | Source ↗ | |
| 12 | GPT-4o (Aug 2024) OpenAI | 26.0% | — | 6 Aug 2024 | — | Source ↗ | |
| 13 | GPT-4.1 Mini OpenAI | 16.0% | — | 14 Apr 2025 | — | Source ↗ | |
| 14 | Claude 3 Haiku Anthropic | 12.0% | — | 7 Mar 2024 | — | Source ↗ |
Caveats: settings (reasoning effort, agent scaffold) differ between rows and materially affect scores; the best reported setting per model is shown. Source: Epoch AI, CC BY 4.0.