–

heretical · No library run 1

32.1Library score
Pass rate89.3%
Simplicity40.4
Problems $$6.05

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE92.7%42.2$2.00
36.2
GLM 5.3 Flash · mini-SWE84.0%43.2$2.55
30.7
GPT-5.6 Luna · Codex91.2%35.7$1.50
29.6
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
note-import25.636.320.427.4techdocs-render41.256.829.842.6editor-import23.418.915.219.2attachment-paste45.041.243.343.2trusted-markdown35.730.632.232.8diagram-preview35.921.731.029.5matrix-message52.652.349.151.4notebook-output33.055.335.441.2email-render20.714.131.722.2svg-ingest52.330.118.433.6collection-render21.211.919.617.6api-docs15.820.417.417.9post-enrich41.58.439.129.7learning-outcome33.832.822.229.6package-panel23.026.217.322.2poll-export60.937.350.149.4reader-extract41.122.630.731.4article-extract70.336.444.250.3timeline-render26.429.730.829.0brand-svg31.625.826.327.9telegram-notify33.830.021.528.4story-markdown32.035.824.730.8