–

heretical · No library run 1

32.1Library score
Pass rate89.3%
Simplicity40.4
Problems $$6.05

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE92.7%42.2$2.00
36.2
GLM 5.3 Flash · mini-SWE84.0%43.2$2.55
30.7
GPT-5.6 Luna · Codex91.2%35.7$1.50
29.6
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.