–

npr · No library run 3

40.1Library score
Pass rate85.7%
Simplicity49.4
Problems $$4.33

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE89.5%49.9$0.94
40.8
GLM 5.3 Flash · mini-SWE80.4%48.6$2.51
38.1
GPT-5.6 Luna · Codex87.1%49.7$0.88
41.4
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.