Kimi K3

44.0± 1.0 (95% CI)

mini-SWE · pass rate 86.0% · simplicity 58.6 · $8.70 per library

Runs

Each cell is one full run: the model designs the library, then 3 implementers solve every problem with it. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 35.4 41.7 34.637.2 ± 3.9

clirs 50.8 59.2 56.355.4 ± 4.3

heretical 42.4 42.4 46.543.8 ± 2.3

npr 54.4 50.5 50.851.9 ± 2.1

plaid 38.6 40.4 36.238.4 ± 2.1

pyda 52.5 50.9 50.051.1 ± 1.3

canon 50.7 56.8 56.354.6 ± 3.4

roadkill 15.0 11.9 16.014.3 ± 2.1

rubberband 69.1 63.1 67.666.6 ± 3.1

weft 36.1 37.1 36.136.5 ± 0.6

sapi 38.4 28.5 39.635.5 ± 6.1

simu 40.2 45.0 35.940.4 ± 4.5

uglypie 54.1 56.4 58.956.4 ± 2.4

umami 45.8 47.5 51.348.2 ± 2.8

wapi 33.3 26.8 29.729.9 ± 3.3