–

npr · No library run 1

40.6Library score
Pass rate84.6%
Simplicity49.8
Problems $$4.27

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE94.4%48.6$0.98
44.2
GLM 5.3 Flash · mini-SWE77.2%52.5$2.40
41.2
GPT-5.6 Luna · Codex82.2%48.2$0.90
36.3
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.