GLM 5.3 Flash

35.0± 5.2 (95% CI)

mini-SWE · pass rate 87.3% · simplicity 47.2

Runs

Each cell is one full attempt: GLM 5.3 Flash solves every problem with the task's production library. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 32.3 30.8 33.432.2 ± 1.3

clirs 51.4 44.9 52.149.5 ± 4.0

heretical 30.0 31.9 29.130.3 ± 1.4

npr 38.2 32.8 27.632.8 ± 5.3

plaid 32.1 33.6 30.131.9 ± 1.8

pyda 50.5 46.5 45.047.3 ± 2.8

canon 33.9 36.8 41.237.3 ± 3.7

roadkill 12.8 12.1 12.712.5 ± 0.4

rubberband 39.9 43.4 39.340.8 ± 2.2

weft 25.4 31.7 26.227.7 ± 3.5

sapi 23.8 27.3 29.326.8 ± 2.8

simu 34.3 32.1 31.432.6 ± 1.5

uglypie 40.5 47.1 48.245.3 ± 4.2

umami 48.8 49.5 44.747.7 ± 2.6

wapi 24.5 24.2 18.822.5 ± 3.2