–

clirs · No library run 2

31.2Library score
Pass rate90.5%
Simplicity37.5
Problems $$2.13

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE89.9%38.4$0.62
31.9
GLM 5.3 Flash · mini-SWE88.8%40.0$0.64
32.1
GPT-5.6 Luna · Codex92.7%34.3$0.87
29.7
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.