–

arazu · No library run 1

22.0Library score
Pass rate76.3%
Simplicity41.5
Problems $$11.58

Implementers get no library and write everything from scratch.

ImplementerPass rateSimplicityCostScore
DeepSeek V4.1 Flash · mini-SWE80.4%35.4$1.81
22.9
GLM 5.3 Flash · mini-SWE73.3%45.9$6.76
19.7
GPT-5.6 Luna · Codex74.7%43.7$3.00
23.5
ProblemDeepSeek V4.1 FlashGLM 5.3 FlashGPT-5.6 LunaMean
lander-lidar31.60.035.122.2scene-contact18.00.015.011.0world-contact14.235.939.629.9aircraft-raycast21.931.325.226.1capsule-training15.113.513.814.2physics-tick11.30.013.58.3offset-strike18.515.816.917.1layered-laser31.822.931.628.8swept-cargo11.00.012.57.8compound-scanner19.429.119.722.7sensor-transit-ledger37.332.639.436.4articulated-creature22.80.00.57.7cable-replay0.023.50.07.8damping35.641.328.335.1one-way36.516.045.832.8material-braking20.225.916.420.8platformer-ccd34.30.036.523.6grab-replay27.241.319.429.3keyed-platform-drop31.844.822.633.1ccd-wall17.838.123.026.3motor-target15.017.027.119.7rope-cut31.54.534.323.4