–

canon · No library run 3

35.1Library score
Pass rate95.0%
Simplicity38.4
Problems $$2.05

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE98.3%38.8$0.62
37.5
GLM 5.3 Flash · mini-SWE92.1%41.0$0.82
35.2
GPT-5.6 Luna · Codex94.8%35.4$0.62
32.8
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
profile-patch29.638.09.325.7thermostat-codec36.310.830.125.7tagged-config43.044.843.443.7rtl-pin-lock35.037.033.435.1pipeline-provenance18.822.222.121.1boundary-views28.427.525.827.2ml-run-config63.852.542.052.8ha-triggers48.755.247.250.4env-config46.945.946.846.5paper-poster54.448.657.253.4typed-client33.827.415.625.6cli-dispatch21.026.824.724.2trading-events42.632.734.836.7llm-contracts22.322.727.124.1