–

clirs · No library run 2

31.2Library score
Pass rate90.5%
Simplicity37.5
Problems $$2.13

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE89.9%38.4$0.62
31.9
GLM 5.3 Flash · mini-SWE88.8%40.0$0.64
32.1
GPT-5.6 Luna · Codex92.7%34.3$0.87
29.7
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
maintainer-tools33.445.033.937.4turbo-query31.919.822.724.8onboard43.937.544.642.0chain-exec8.214.037.019.7subcommand-groups17.832.931.427.3pointer-indexer25.630.527.427.8site-builder46.430.529.735.5launcher-plan35.035.120.130.1process-inspect33.631.740.635.3relay-compiler32.729.524.829.0cross-build27.316.914.819.7backup-variants38.047.921.635.8devctl-families40.645.737.641.3