Leaderboard
17 tasks / 4 runs per task / September 2026
17 tasks4 runs per taskSeptember 2026
Results
Each configuration is evaluated four times per task.
| Task details | Model | Effort | Agent | ||||
|---|---|---|---|---|---|---|---|
gpt-6-sol Sep 24 | xhigh | codex 0.156.1 | 69.1% ± 10.1% | 51.2 | 50.2 | $90.86 | |
gpt-6-astra Sep 08 | xhigh | codex 0.153.4 | 64.7% ± 4.8% | 31.3 | 30.3 | $325.01 | |
gpt-6-sol Sep 24 | high | codex 0.156.1 | 61.8% ± 5.9% | 42.6 | 41.6 | $62.65 | |
gpt-6-astra Sep 08 | medium | codex 0.153.4 | 55.9% ± 5.9% | 22.3 | 21.3 | $129.31 | |
gpt-6-astra Sep 08 | high | codex 0.153.4 | 54.4% ± 5.6% | 24.7 | 23.7 | $191.36 | |
gpt-5.6-sol Sep 08 | xhigh | codex 0.153.4 | 53.7% ± 12.8% | 92.9 | 91.7 | $432.48 | |
gpt-6-sol Sep 24 | medium | codex 0.156.1 | 52.9% ± 0.0% | 38.7 | 37.7 | $46.16 | |
gpt-6-astra Sep 08 | low | codex 0.153.4 | 50.0% ± 3.4% | 18.8 | 17.8 | $92.37 | |
gpt-5.6-sol Sep 08 | high | codex 0.153.4 | 48.5% ± 5.6% | 61.2 | 60.1 | $260.95 | |
gpt-6-sol Sep 24 | low | codex 0.156.1 | 45.6% ± 12.1% | 22.6 | 21.6 | $19.18 | |
deepseek-flash Sep 24 | default | claude-code 2.1.278 | 42.6% ± 2.9% | 226.4 | 285.1 | $60.73 | |
gpt-6-luna Sep 24 | xhigh | codex 0.156.1 | 42.6% ± 5.6% | 29.5 | 28.5 | $3.52 | |
gpt-5.6-sol Sep 08 | medium | codex 0.153.4 | 41.2% ± 8.3% | 35.8 | 34.8 | $105.78 | |
gpt-5.6-sol Sep 08 | low | codex 0.153.4 | 36.8% ± 11.1% | 23.5 | 22.5 | $57.35 | |
gpt-6-luna Sep 24 | high | codex 0.156.1 | 35.3% ± 4.8% | 20.6 | 19.6 | $1.21 | |
gpt-6-luna Sep 24 | medium | codex 0.156.1 | 20.6% ± 11.3% | 14.1 | 13.1 | $0.58 | |
gpt-6-luna Sep 24 | low | codex 0.156.1 | 5.9% ± 0.0% | 26.5 | 25.5 | $1.27 |