–
umami · No library run 3
59.2Library score
Pass rate95.5%
Simplicity68.1
Problems $$3.20
Implementers get no library and write everything from scratch.
ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE96.1%78.7$0.80
GLM 5.3 Flash · mini-SWE95.8%69.2$1.42
GPT-5.6 Luna · Codex94.6%56.3$0.98
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
Click a problem to see how each solution's score is built.