–

umami · No library run 2

61.4Library score
Pass rate96.3%
Simplicity68.0
Problems $$3.19

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE98.9%77.6$0.88
72.8
GLM 5.3 Flash · mini-SWE96.4%71.9$1.35
64.5
GPT-5.6 Luna · Codex93.6%54.5$0.95
46.9
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
db-notifications75.255.254.461.6log-worker-pool78.153.736.256.0backup-transfer27.422.828.626.3session-mailboxes52.556.946.552.0logger-shutdown54.650.031.045.2transport-ingress81.882.243.969.3server-output95.871.189.685.5packet-colors67.763.954.161.9client-routing84.784.457.675.6request-reservation68.762.220.050.3sync-progress69.956.233.853.3game-event-lanes81.988.976.782.5ui-event-bridge88.484.862.478.5network-bridge73.466.640.860.2logger-drain59.957.835.150.9discord-subscriptions98.252.854.268.4game-session65.670.451.562.5consumer-flow80.081.236.365.9chain-sync-poller73.254.739.555.8editor-commands79.173.546.866.4