DeepSeek V4 Flash

34.8± 6.4 (95% CI)

mini-SWE · pass rate 88.0% · simplicity 49.3

Runs

Each cell is one full attempt: DeepSeek V4 Flash solves every problem with the task's production library. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 15.4 16.6 13.315.1 ± 1.7

clirs 52.5 41.5 52.048.7 ± 6.2

heretical 25.8 23.4 26.325.2 ± 1.5

npr 37.4 33.4 30.533.8 ± 3.5

plaid 31.6 27.7 33.831.0 ± 3.1

pyda 45.1 44.4 41.543.7 ± 1.9

canon 44.9 45.0 46.145.3 ± 0.7

roadkill 14.8 20.8 21.919.2 ± 3.8

rubberband 32.7 35.6 39.235.9 ± 3.3

weft 26.2 33.1 24.928.1 ± 4.4

sapi 37.7 38.1 35.737.2 ± 1.3

simu 31.9 35.1 32.933.3 ± 1.6

uglypie 47.5 44.0 54.248.6 ± 5.2

umami 52.8 55.9 54.654.4 ± 1.6

wapi 29.7 34.3 32.832.3 ± 2.3