–

sapi · No library run 1

20.4Library score
Pass rate94.9%
Simplicity22.6
Problems $$1.10

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE96.8%23.3$0.24
21.7
GLM 5.3 Flash · mini-SWE96.0%22.4$0.36
20.6
GPT-5.6 Luna · Codex91.8%22.2$0.51
19.0
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.