–
clirs · No library run 3
29.4Library score
Pass rate88.1%
Simplicity36.5
Problems $$2.07
Implementers get no library and write everything from scratch.
ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE81.3%35.1$0.64
GLM 5.3 Flash · mini-SWE89.8%39.4$0.61
GPT-5.6 Luna · Codex93.2%35.0$0.82
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
Click a problem to see how each solution's score is built.