–

simu · No library run 1

39.1Library score
Pass rate86.7%
Simplicity53.4
Problems $$2.97

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE84.9%52.7$1.02
38.0
GLM 5.3 Flash · mini-SWE90.6%56.4$1.15
39.8
GPT-5.6 Luna · Codex85.3%51.5$0.79
39.3
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
intermodal-terminal-flow30.035.032.132.4diagnostic-specimen-release45.036.943.041.6rail-corridor-recovery44.783.786.571.6binary-offload-episode49.855.061.055.3packet-schedulers35.143.739.139.3wildfire-relief-depot15.20.016.510.6preempted-fault-race46.043.632.940.8rendezvous-rollback25.262.121.536.3filter-reservation46.443.525.338.4one-shot-bell64.254.462.360.3quorum-poisoned-losers46.841.748.245.5harbor-surge19.117.414.817.1harbor-vessels27.20.028.118.4