Fable 5.1

47.5± 1.1 (95% CI)

mini-SWE · pass rate 86.1% · simplicity 62.7 · $14.81 per library

Runs

Each cell is one full run: the model designs the library, then 3 implementers solve every problem with it. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 38.3 39.0 31.336.2 ± 4.2

clirs 55.9 68.5 52.959.1 ± 8.3

heretical 48.5 50.8 53.050.8 ± 2.2

npr 48.9 53.0 49.450.4 ± 2.3

plaid 45.7 45.8 47.846.5 ± 1.2

pyda 54.9 54.5 56.855.4 ± 1.2

canon 59.4 61.2 61.660.7 ± 1.2

roadkill 13.9 15.9 15.715.1 ± 1.1

rubberband 74.6 77.3 76.476.1 ± 1.4

weft 36.6 35.0 38.536.7 ± 1.8

sapi 43.4 41.3 48.144.3 ± 3.5

simu 43.6 42.4 40.342.1 ± 1.7

uglypie 55.4 59.6 56.557.2 ± 2.2

umami 48.1 48.4 50.749.1 ± 1.4

wapi 32.8 34.3 29.432.2 ± 2.5