–

clirs · No library run 1

30.8Library score
Pass rate89.2%
Simplicity37.8
Problems $$2.06

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE93.6%37.8$0.60
33.2
GLM 5.3 Flash · mini-SWE81.0%41.2$0.61
29.2
GPT-5.6 Luna · Codex93.2%34.4$0.85
30.0
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
maintainer-tools41.335.937.638.3turbo-query19.827.520.022.4onboard42.141.937.940.6chain-exec35.510.734.927.0subcommand-groups29.736.134.533.4pointer-indexer26.128.021.925.3site-builder38.635.028.634.1launcher-plan31.834.129.231.7process-inspect34.20.027.120.4relay-compiler29.931.323.428.2cross-build25.429.717.524.2backup-variants39.835.536.437.2devctl-families37.734.640.637.6