Sonnet 5.5

63.8± 7.0 (95% CI)

mini-SWE · pass rate 91.8% · simplicity 75.7

Runs

Each cell is one full attempt: Sonnet 5.5 solves every problem with the task's production library. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 61.3 62.7 61.261.7 ± 0.8

clirs 76.0 77.8 77.577.1 ± 1.0

heretical 47.0 48.1 49.648.2 ± 1.3

npr 62.0 65.9 62.963.6 ± 2.0

plaid 78.1 76.5 78.377.7 ± 1.0

pyda 74.9 75.9 70.873.9 ± 2.7

canon 70.4 68.2 69.869.5 ± 1.1

roadkill 31.6 38.1 31.433.7 ± 3.8

rubberband 68.6 67.5 68.568.2 ± 0.6

weft 70.8 65.3 69.068.4 ± 2.8

sapi 43.9 41.9 43.443.1 ± 1.1

simu 54.9 53.0 56.454.8 ± 1.7

uglypie 69.2 69.6 67.068.6 ± 1.4

umami 83.7 83.3 79.282.0 ± 2.5

wapi 57.6 63.8 64.161.8 ± 3.7