–

umami · No library run 1

57.0Library score
Pass rate94.2%
Simplicity65.5
Problems $$3.31

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE97.4%75.2$0.89
68.8
GLM 5.3 Flash · mini-SWE92.6%65.4$1.40
55.8
GPT-5.6 Luna · Codex92.7%56.0$1.02
46.5
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
db-notifications73.054.866.864.8log-worker-pool62.362.960.661.9backup-transfer30.024.338.931.1session-mailboxes55.654.343.551.1logger-shutdown51.837.231.740.2transport-ingress89.375.046.070.1server-output84.664.064.571.0packet-colors81.854.337.757.9client-routing80.186.255.974.1request-reservation70.222.622.538.4sync-progress68.050.632.150.2game-event-lanes88.963.871.174.6ui-event-bridge89.076.349.471.6network-bridge81.464.851.966.1logger-drain51.320.133.034.8discord-subscriptions100.067.548.872.1game-session67.632.848.949.8consumer-flow32.187.241.353.5chain-sync-poller40.046.142.742.9editor-commands78.571.343.164.3