P

sapi · Production library run 1

49.7Library score
Pass rate97.0%
Simplicity52.6
Problems $$2.31

Implementers use the real production library (fastapi).

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE96.8%56.6$0.52
53.1
GLM 5.3 Flash · mini-SWE96.9%56.1$0.89
53.5
GPT-5.6 Luna · Codex97.2%45.0$0.90
42.6
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
fleet-api63.167.028.953.0telemetry-api56.868.540.955.4catalog-search48.457.345.750.5delivery-profile40.958.161.553.5shipment-preview55.245.334.545.0protected-notes26.027.635.729.8sensor-export63.358.933.651.9versioned-orders52.763.751.856.1browser-sessions40.924.131.332.1signed-webhook67.354.358.760.1service-catalog69.363.345.659.4