–
umami · No library run 2
61.4Library score
Pass rate96.3%
Simplicity68.0
Problems $$3.19
Implementers get no library and write everything from scratch.
ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE98.9%77.6$0.88
GLM 5.3 Flash · mini-SWE96.4%71.9$1.35
GPT-5.6 Luna · Codex93.6%54.5$0.95
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
Click a problem to see how each solution's score is built.