No design phase. Each agent gets a real production library and one instruction: use it, and write as little code as possible.

LeaderboardParetoExtended results →
ModelScore ± 95% CI$ / problem
Opus 5.5mini-SWE
Opus 5.566.9, 95% CI 59.8 to 74.1
66.9 ± 7.1
$0.664
Opus 5mini-SWE
Opus 555.1, 95% CI 48.4 to 61.8
55.1 ± 6.7
$1.314
DeepSeek V4.1 Flashmini-SWE
DeepSeek V4.1 Flash43.8, 95% CI 38.5 to 49.1
43.8 ± 5.3
$0.190
GPT-5.6 Terramini-SWE
GPT-5.6 Terra40.0, 95% CI 33.4 to 46.6
40.0 ± 6.6
$0.239
Sonnet 5mini-SWE
Sonnet 539.9, 95% CI 33.1 to 46.6
39.9 ± 6.8
$0.919
GLM 5.3 Flashmini-SWE
GLM 5.3 Flash35.0, 95% CI 29.8 to 40.2
35.0 ± 5.2
$0.192
DeepSeek V4 Flashmini-SWE
DeepSeek V4 Flash34.8, 95% CI 28.4 to 41.1
34.8 ± 6.4
$0.286
GPT-5.6 Lunamini-SWE
GPT-5.6 Luna30.1, 95% CI 24.3 to 35.9
30.1 ± 5.8
$0.042
Muse Spark 1.3mini-SWE
Muse Spark 1.329.4, 95% CI 23.9 to 35.0
29.4 ± 5.6
$1.940
Note: Scores run 0–100: pass rate² × simplicity, averaged over every problem, with problems weighted equally. The error bars and ± show the 95% confidence interval.