Opus 5.5

48.9± 0.9 (95% CI)

mini-SWE · pass rate 86.6% · simplicity 64.5 · $9.62 per library

Runs

Each cell is one full run: the model designs the library, then 3 implementers solve every problem with it. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 44.2 39.9 43.242.4 ± 2.2

clirs 67.7 66.6 57.764.0 ± 5.5

heretical 46.6 50.3 47.848.3 ± 1.9

npr 63.0 51.6 52.655.7 ± 6.3

plaid 46.9 47.4 45.546.6 ± 1.0

pyda 57.7 60.5 60.459.5 ± 1.6

canon 64.3 66.0 68.366.2 ± 2.0

roadkill 18.1 17.8 15.117.0 ± 1.6

rubberband 77.0 74.1 79.977.0 ± 2.9

weft 33.7 34.0 38.035.3 ± 2.4

sapi 41.8 41.8 42.341.9 ± 0.3

simu 40.1 38.8 38.239.1 ± 1.0

uglypie 59.2 55.0 61.558.6 ± 3.3

umami 50.1 48.1 48.849.0 ± 1.0

wapi 32.0 34.8 33.033.3 ± 1.4