–

clirs · No library run 3

29.4Library score
Pass rate88.1%
Simplicity36.5
Problems $$2.07

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE81.3%35.1$0.64
25.9
GLM 5.3 Flash · mini-SWE89.8%39.4$0.61
32.0
GPT-5.6 Luna · Codex93.2%35.0$0.82
30.4
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
maintainer-tools31.541.327.133.3turbo-query18.521.119.619.7onboard42.336.441.740.1chain-exec37.041.839.139.3subcommand-groups16.034.725.525.4pointer-indexer6.026.527.920.2site-builder32.846.530.336.5launcher-plan32.931.134.532.8process-inspect29.533.133.332.0relay-compiler27.523.924.125.1cross-build0.020.821.314.0backup-variants27.033.531.330.6devctl-families35.825.139.833.6