P

sapi · Production library run 3

47.5Library score
Pass rate96.9%
Simplicity50.2
Problems $$2.72

Implementers use the real production library (fastapi).

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE98.0%55.7$0.55
53.7
GLM 5.3 Flash · mini-SWE97.2%50.1$1.22
47.7
GPT-5.6 Luna · Codex95.6%44.7$0.95
41.0
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
fleet-api61.562.730.051.4telemetry-api69.962.050.260.7catalog-search64.545.344.651.5delivery-profile61.467.533.954.3shipment-preview34.539.226.933.5protected-notes30.533.829.431.3sensor-export75.137.144.852.4versioned-orders44.436.554.745.2browser-sessions32.631.431.531.8signed-webhook70.651.559.660.6service-catalog46.257.745.649.8