–

umami · No library run 3

59.2Library score
Pass rate95.5%
Simplicity68.1
Problems $$3.20

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE96.1%78.7$0.80
69.0
GLM 5.3 Flash · mini-SWE95.8%69.2$1.42
58.4
GPT-5.6 Luna · Codex94.6%56.3$0.98
50.1
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
db-notifications74.676.865.072.1log-worker-pool48.473.740.454.2backup-transfer44.90.036.127.0session-mailboxes44.948.936.443.4logger-shutdown70.232.734.145.7transport-ingress68.549.468.062.0server-output71.968.367.369.2packet-colors94.367.648.570.1client-routing70.365.940.558.9request-reservation26.954.719.133.6sync-progress72.261.735.156.3game-event-lanes80.784.671.578.9ui-event-bridge88.690.973.884.4network-bridge82.055.062.966.6logger-drain67.741.344.251.1discord-subscriptions71.355.056.160.8game-session68.253.251.857.8consumer-flow71.580.157.769.8chain-sync-poller81.452.745.059.7editor-commands82.356.348.062.2