Opus 5

55.1± 6.7 (95% CI)

mini-SWE · pass rate 87.0% · simplicity 68.5

Runs

Each cell is one full attempt: Opus 5 solves every problem with the task's production library. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 57.1 61.6 54.157.6 ± 3.8

clirs 58.7 54.1 65.459.4 ± 5.7

heretical 43.2 41.2 41.541.9 ± 1.1

npr 52.8 53.9 51.852.9 ± 1.0

plaid 63.8 66.5 67.465.9 ± 1.9

pyda 66.6 63.6 63.564.6 ± 1.8

canon 69.7 66.4 69.868.6 ± 1.9

roadkill 39.9 40.3 43.541.2 ± 2.0

rubberband 62.0 60.7 59.560.7 ± 1.2

weft 32.1 32.6 27.530.7 ± 2.8

sapi 39.8 44.5 39.941.4 ± 2.7

simu 60.1 57.4 57.258.2 ± 1.6

uglypie 64.8 63.2 65.764.5 ± 1.3

umami 74.2 76.6 73.474.7 ± 1.7

wapi 26.7 30.5 26.728.0 ± 2.2