–

sapi · No library run 2

19.1Library score
Pass rate92.3%
Simplicity21.9
Problems $$1.30

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE97.6%22.2$0.38
21.1
GLM 5.3 Flash · mini-SWE81.6%21.8$0.38
15.7
GPT-5.6 Luna · Codex97.6%21.7$0.54
20.4
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
fleet-api20.614.019.017.9telemetry-api28.80.025.918.2catalog-search15.214.118.616.0delivery-profile18.311.314.514.7shipment-preview11.012.311.911.7protected-notes23.320.823.022.4sensor-export21.016.320.419.3versioned-orders18.111.819.116.3browser-sessions24.422.723.923.6signed-webhook36.733.934.735.1service-catalog15.015.513.814.8