–

canon · No library run 2

35.2Library score
Pass rate93.9%
Simplicity38.5
Problems $$2.01

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE99.3%38.6$0.62
38.1
GLM 5.3 Flash · mini-SWE92.0%39.7$0.74
34.9
GPT-5.6 Luna · Codex90.5%37.3$0.66
32.5
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.