Leaderboard
17 tasks / 4 runs per task / September 2026
17 tasks4 runs per task
Results
Each configuration is evaluated four times per task.
| Task details | Model | Effort | Agent | ||||
|---|---|---|---|---|---|---|---|
gpt-6-astra | xhigh | codex 0.153.4 | 64.7% ± 4.8% | 31.3 | 30.3 | $325.01 | |
gpt-6-astra | medium | codex 0.153.4 | 55.9% ± 5.9% | 22.3 | 21.3 | $129.31 | |
gpt-6-astra | high | codex 0.153.4 | 54.4% ± 5.6% | 24.7 | 23.7 | $191.36 | |
gpt-5.6-sol | xhigh | codex 0.153.4 | 53.7% ± 12.8% | 92.9 | 91.7 | $432.48 | |
gpt-6-astra | low | codex 0.153.4 | 50.0% ± 3.4% | 18.8 | 17.8 | $92.37 | |
gpt-5.6-sol | high | codex 0.153.4 | 48.5% ± 5.6% | 61.2 | 60.1 | $260.95 | |
gpt-5.6-sol | medium | codex 0.153.4 | 41.2% ± 8.3% | 35.8 | 34.8 | $105.78 | |
gpt-5.6-sol | low | codex 0.153.4 | 36.8% ± 11.1% | 23.5 | 22.5 | $57.35 |