Sonnet 5

39.9± 6.8 (95% CI)

mini-SWE · pass rate 80.1% · simplicity 56.0

Runs

Each cell is one full attempt: Sonnet 5 solves every problem with the task's production library. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 34.9 38.1 36.636.5 ± 1.6

clirs 41.0 37.1 41.239.8 ± 2.3

heretical 37.7 36.1 37.237.0 ± 0.8

npr 38.5 41.7 35.338.5 ± 3.2

plaid 45.5 45.2 46.045.6 ± 0.4

pyda 48.0 50.5 46.048.2 ± 2.3

canon 43.1 37.2 43.541.3 ± 3.5

roadkill 20.3 20.2 22.921.1 ± 1.6

rubberband 56.2 49.5 54.953.5 ± 3.6

weft 18.2 18.0 19.718.6 ± 0.9

sapi 32.5 32.1 30.331.6 ± 1.2

simu 36.9 35.4 33.435.3 ± 1.8

uglypie 53.8 52.1 49.151.7 ± 2.3

umami 66.3 62.4 65.864.8 ± 2.1

wapi 12.3 12.1 15.313.2 ± 1.8