–

heretical · No library run 3

32.6Library score
Pass rate90.6%
Simplicity39.4
Problems $$7.13

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE94.0%40.4$2.18
35.5
GLM 5.3 Flash · mini-SWE84.5%42.6$3.51
31.6
GPT-5.6 Luna · Codex93.3%35.2$1.44
30.7
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.