–

canon · No library run 1

34.1Library score
Pass rate93.7%
Simplicity37.6
Problems $$2.06

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE97.7%37.2$0.62
35.8
GLM 5.3 Flash · mini-SWE94.9%39.7$0.76
35.7
GPT-5.6 Luna · Codex88.4%35.9$0.68
30.6
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.