–

npr · No library run 1

40.6Library score
Pass rate84.6%
Simplicity49.8
Problems $$4.27

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE94.4%48.6$0.98
44.2
GLM 5.3 Flash · mini-SWE77.2%52.5$2.40
41.2
GPT-5.6 Luna · Codex82.2%48.2$0.90
36.3
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
cluster-consensus29.514.133.025.6logistic-path68.60.087.352.0relu-regions11.00.020.810.6table-reasoning43.057.975.058.6embedding-pooling51.262.945.753.3kv-cache46.344.650.147.0channel-stft55.066.352.057.8irls-path60.355.544.053.2peak-covariance33.51.326.920.6envelope-curvature66.979.844.963.9radiance-moments65.168.63.245.6kalman-update34.256.243.244.5csi-pose44.50.00.014.8vmc-update0.160.40.120.2pose-correction52.558.24.238.3spectral-reconstruction26.827.623.025.8portfolio-risk51.968.838.953.2contact-forces60.436.538.045.0density-propagation36.75.045.128.9tensor-train46.161.351.352.9