DeepSeek V4.1 Flash

43.8± 5.3 (95% CI)

mini-SWE · pass rate 90.5% · simplicity 52.1

Runs

Each cell is one full attempt: DeepSeek V4.1 Flash solves every problem with the task's production library. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 35.9 36.7 34.935.9 ± 0.9

clirs 42.3 52.2 57.950.8 ± 7.9

heretical 37.8 37.8 40.638.7 ± 1.6

npr 40.6 43.0 42.141.9 ± 1.2

plaid 44.8 44.3 44.044.3 ± 0.4

pyda 57.3 59.2 59.058.5 ± 1.0

canon 45.5 45.0 48.446.3 ± 1.8

roadkill 25.1 25.7 23.624.8 ± 1.1

rubberband 47.3 45.5 52.948.6 ± 3.9

weft 35.4 32.9 33.934.1 ± 1.3

sapi 30.1 31.1 36.132.4 ± 3.2

simu 39.0 42.3 38.940.1 ± 2.0

uglypie 47.8 46.7 53.449.3 ± 3.6

umami 61.2 59.9 59.960.3 ± 0.8

wapi 50.8 44.4 38.344.5 ± 6.2