–

npr · No library run 3

40.1Library score
Pass rate85.7%
Simplicity49.4
Problems $$4.33

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE89.5%49.9$0.94
40.8
GLM 5.3 Flash · mini-SWE80.4%48.6$2.51
38.1
GPT-5.6 Luna · Codex87.1%49.7$0.88
41.4
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
cluster-consensus30.014.131.225.1logistic-path86.762.161.970.3relu-regions12.316.126.118.1table-reasoning59.20.089.749.6embedding-pooling51.163.839.051.3kv-cache43.847.450.447.2channel-stft53.160.251.154.8irls-path57.861.649.456.2peak-covariance28.11.11.110.1envelope-curvature63.48.567.246.4radiance-moments8.759.056.541.4kalman-update44.445.835.441.9csi-pose44.256.87.136.0vmc-update0.162.00.120.8pose-correction58.210.852.440.5spectral-reconstruction12.225.328.622.1portfolio-risk49.264.641.451.8contact-forces29.341.036.735.7density-propagation38.73.746.429.6tensor-train46.058.555.853.4