P

canon · Production library run 3

59.1Library score
Pass rate91.2%
Simplicity68.3
Problems $$4.76

Implementers use the real production library (pydantic).

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE96.2%73.5$1.03
68.5
GLM 5.3 Flash · mini-SWE90.5%67.7$2.39
57.9
GPT-5.6 Luna · Codex86.7%63.8$1.34
50.7
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
profile-patch93.479.02.358.2thermostat-codec64.365.010.046.4tagged-config73.881.675.176.8rtl-pin-lock94.60.965.653.7pipeline-provenance95.056.460.570.6boundary-views61.347.166.458.3ml-run-config89.986.285.687.2ha-triggers61.777.874.671.4env-config85.179.169.377.8paper-poster58.046.744.749.8typed-client53.039.345.746.0cli-dispatch35.136.731.234.3trading-events66.974.052.164.3llm-contracts26.941.427.231.9