–

heretical · No library run 2

32.1Library score
Pass rate90.9%
Simplicity39.3
Problems $$6.45

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE94.0%40.6$2.10
35.9
GLM 5.3 Flash · mini-SWE86.9%40.7$2.90
29.5
GPT-5.6 Luna · Codex91.6%36.7$1.46
30.8
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
note-import34.937.621.931.5techdocs-render49.00.027.325.4editor-import29.813.130.524.5attachment-paste54.940.038.144.3trusted-markdown27.135.832.431.8diagram-preview39.019.429.729.4matrix-message65.144.160.656.6notebook-output47.657.754.353.2email-render23.815.931.523.7svg-ingest37.035.019.330.4collection-render25.236.816.826.2api-docs16.412.116.214.9post-enrich50.47.340.632.8learning-outcome40.043.930.037.9package-panel20.930.916.622.8poll-export54.147.455.852.4reader-extract20.621.319.220.4article-extract40.928.538.536.0timeline-render33.535.232.433.7brand-svg27.027.428.527.6telegram-notify32.128.718.826.5story-markdown20.832.018.423.8