–

heretical · No library run 3

32.6Library score
Pass rate90.6%
Simplicity39.4
Problems $$7.13

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE94.0%40.4$2.18
35.5
GLM 5.3 Flash · mini-SWE84.5%42.6$3.51
31.6
GPT-5.6 Luna · Codex93.3%35.2$1.44
30.7
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
note-import33.036.120.829.9techdocs-render39.640.031.036.9editor-import19.514.027.820.5attachment-paste36.938.150.341.8trusted-markdown37.518.129.528.4diagram-preview24.616.832.524.6matrix-message49.048.945.447.8notebook-output47.857.146.250.4email-render34.028.932.831.9svg-ingest33.621.124.426.4collection-render20.026.315.820.7api-docs21.120.817.919.9post-enrich49.138.349.545.6learning-outcome55.551.221.342.7package-panel16.335.318.923.5poll-export59.961.558.259.9reader-extract33.623.418.725.2article-extract45.237.128.236.8timeline-render27.035.021.227.7brand-svg32.00.030.921.0telegram-notify34.424.821.226.8story-markdown31.121.732.328.4