–

simu · No library run 3

40.9Library score
Pass rate85.8%
Simplicity53.4
Problems $$3.55

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE87.7%53.6$1.01
43.4
GLM 5.3 Flash · mini-SWE89.1%58.1$1.77
46.6
GPT-5.6 Luna · Codex80.5%48.5$0.77
32.7
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
intermodal-terminal-flow29.034.439.134.2diagnostic-specimen-release46.644.735.242.2rail-corridor-recovery88.172.538.866.5binary-offload-episode57.463.352.457.7packet-schedulers36.448.132.238.9wildfire-relief-depot14.521.420.518.8preempted-fault-race42.454.728.041.7rendezvous-rollback73.755.97.945.8filter-reservation46.144.534.041.5one-shot-bell58.054.156.756.2quorum-poisoned-losers47.143.046.245.4harbor-surge16.730.28.618.5harbor-vessels8.539.125.124.2