GPT-6 Astra

44.1± 0.8 (95% CI)

Codex · pass rate 85.5% · simplicity 58.5 · $4.51 per library

Runs

Each cell is one full run: the model designs the library, then 3 implementers solve every problem with it. Click a cell to open that run.

TaskRun 1Run 2Run 3Mean ± std

arazu 43.0 43.7 42.243.0 ± 0.7

clirs 58.8 60.4 59.559.6 ± 0.8

heretical 36.5 40.8 39.338.9 ± 2.2

npr 52.4 54.4 49.652.1 ± 2.5

plaid 46.5 46.3 46.746.5 ± 0.2

pyda 52.9 50.6 50.351.3 ± 1.4

canon 60.0 59.0 55.658.2 ± 2.3

roadkill 7.4 9.4 9.78.8 ± 1.2

rubberband 61.8 65.5 59.562.3 ± 3.1

weft 35.1 33.2 29.132.5 ± 3.1

sapi 35.4 39.8 43.039.4 ± 3.8

simu 36.7 40.6 38.038.5 ± 2.0

uglypie 54.9 58.8 57.357.0 ± 1.9

umami 47.1 46.2 46.946.7 ± 0.5

wapi 26.2 32.6 23.627.4 ± 4.6