–

sapi · No library run 1

20.4Library score
Pass rate94.9%
Simplicity22.6
Problems $$1.10

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE96.8%23.3$0.24
21.7
GLM 5.3 Flash · mini-SWE96.0%22.4$0.36
20.6
GPT-5.6 Luna · Codex91.8%22.2$0.51
19.0
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
fleet-api22.125.121.122.8telemetry-api32.232.22.022.1catalog-search20.015.821.319.0delivery-profile14.810.514.213.2shipment-preview13.312.012.212.5protected-notes23.719.123.222.0sensor-export20.720.524.121.8versioned-orders18.421.320.720.1browser-sessions21.820.425.822.7signed-webhook36.935.031.434.5service-catalog14.514.713.114.1