–

umami · No library run 1

57.0Library score
Pass rate94.2%
Simplicity65.5
Problems $$3.31

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE97.4%75.2$0.89
68.8
GLM 5.3 Flash · mini-SWE92.6%65.4$1.40
55.8
GPT-5.6 Luna · Codex92.7%56.0$1.02
46.5
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean

Click a problem to see how each solution's score is built.