GPT-6 Sol

57.6± 6.0 (95% CI)

mini-SWE · pass rate 92.3% · simplicity 66.8

Runs

Each cell is one full attempt: GPT-6 Sol solves every problem with the task's production library. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 53.8 60.7 58.057.5 ± 3.5

clirs 65.9 66.1 64.265.4 ± 1.0

heretical 45.2 46.4 46.145.9 ± 0.7

npr 62.2 59.5 60.760.8 ± 1.3

plaid 67.7 65.1 65.166.0 ± 1.5

pyda 72.0 71.6 71.471.7 ± 0.3

canon 60.8 61.7 68.863.8 ± 4.4

roadkill 26.9 26.2 29.427.5 ± 1.7

rubberband 62.8 59.2 61.561.2 ± 1.8

weft 51.6 50.3 52.851.6 ± 1.2

sapi 47.8 47.8 50.448.7 ± 1.5

simu 54.9 53.1 55.154.4 ± 1.1

uglypie 74.1 69.4 73.672.3 ± 2.6

umami 65.9 65.8 66.165.9 ± 0.2

wapi 45.2 46.1 46.946.1 ± 0.8