P

canon · Production library run 2

59.4Library score
Pass rate91.6%
Simplicity68.9
Problems $$4.85

Implementers use the real production library (pydantic).

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE96.1%71.9$0.98
65.6
GLM 5.3 Flash · mini-SWE94.8%72.3$2.49
65.5
GPT-5.6 Luna · Codex83.9%62.5$1.38
47.1
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
profile-patch77.775.62.151.8thermostat-codec76.992.555.574.9tagged-config93.073.269.778.6rtl-pin-lock91.672.269.877.9pipeline-provenance66.462.868.966.0boundary-views51.968.323.848.0ml-run-config78.385.181.881.8ha-triggers67.075.664.068.9env-config76.869.458.568.2paper-poster55.455.247.752.8typed-client48.046.729.641.4cli-dispatch40.629.633.534.6trading-events73.370.543.762.5llm-contracts22.139.910.724.2