–

roadkill · No library run 3

6.5Library score
Pass rate54.3%
Simplicity26.9
Problems $$5.05

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE53.5%24.8$1.54
6.5
GLM 5.3 Flash · mini-SWE59.4%26.7$2.49
4.6
GPT-5.6 Luna · Codex53.1%29.1$1.03
8.5
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
transit-route-check0.40.01.70.7live-session18.80.08.08.9charging-routes1.80.02.01.3link-time-bins9.48.414.910.9explicit-fleet8.20.06.54.9braess-network5.90.07.14.3signal-plan-sweep4.00.011.85.3merge-spillback3.70.07.33.7incident-guidance4.30.04.52.9corridor-replications0.721.30.07.3speed-limit-control6.40.019.48.6reactive-signals8.921.920.217.0signal-progression4.30.08.84.4gridlock-gating11.30.017.89.7workzone-recovery4.414.10.06.2district-los11.47.25.98.2