–

No library

34.4± 0.5 (95% CI)

Reference arm · pass rate 86.4% · simplicity 46.3

Runs

Each cell is one full attempt: the 3 implementers solve every problem with no library. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 22.0 21.9 23.022.3 ± 0.6

clirs 30.8 31.2 29.430.5 ± 0.9

heretical 32.1 32.1 32.632.3 ± 0.3

npr 40.6 43.1 40.141.3 ± 1.6

plaid 37.4 35.6 36.036.3 ± 1.0

pyda 53.2 50.5 50.151.3 ± 1.7

canon 34.1 35.2 35.134.8 ± 0.6

roadkill 8.0 7.8 6.57.4 ± 0.8

rubberband 40.5 42.4 39.640.8 ± 1.4

weft 34.3 34.6 32.733.9 ± 1.0

sapi 20.4 19.1 20.420.0 ± 0.8

simu 39.1 38.6 40.939.5 ± 1.2

uglypie 32.5 31.7 31.631.9 ± 0.5

umami 57.0 61.4 59.259.2 ± 2.2

wapi 29.4 33.0 40.134.2 ± 5.5