–
npr · No library run 2
43.1Library score
Pass rate90.6%
Simplicity49.7
Problems $$4.96
Implementers get no library and write everything from scratch.
ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE98.8%51.1$0.96
GLM 5.3 Flash · mini-SWE81.7%48.5$3.11
GPT-5.6 Luna · Codex91.4%49.7$0.89
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
Click a problem to see how each solution's score is built.