# GMod Bench report

Run: `c4a59a56-eb25-4438-afae-b13af70d3626`
Fixtures: 38 fixtures (`bun run bench list`)
Repeat: 1 · Concurrency: 4
Started: 2026-07-11T06:50:37.822Z · Completed: 2026-07-11T06:53:24.362Z
Wall clock: 167s · Sum of attempt durations: 653263ms (avg 8596ms, min 5551ms, max 18770ms)

## Model leaderboard

Sorted by **fixture score** (each fixture weighted equally; failures score zero), then fixture solve rate, coverage, and pass rate.

| Rank | Runner / model | Fixture score | Fixture solve | Pass | Partial | Incorrect | Scored | Coverage | Pass rate | Fixture pass | Scored quality | Protocol err | Fixtures | Avg ms | Prompt/Compl tok |
| ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | `agy/Claude Sonnet 4.6 (Thinking)` | 78.9% | 76.3% | 29 | 2 | 4 | 35 | 92.1% | 82.9% | 82.9% | 0.86 | 3 | 29/38 | 9886 | 2761/5312 |
| 2 | `agy/Claude Opus 4.6 (Thinking)` | 27.6% | 26.3% | 10 | 1 | 1 | 12 | 31.6% | 83.3% | 83.3% | 0.88 | 26 | 10/38 | 7306 | 2761/1173 |

## Rates

- Scored attempts: 47
- Pass / partial / incorrect: 39 / 3 / 5
- Pass rate (scored): 39/47
- Fixture pass rate (passed fixture groups / scored fixture groups): 39/47
- Overall mean score (pass=1, partial=0.5): 0.862
- Protocol errors: 29
- Policy violations: 0
- Trace errors: 0
- Timeouts: 0
- Unavailable: 0
- Unsupported: 0

## Usage & timing

Token counts: **provider** when the runner reports them (OpenRouter); otherwise **estimated** (~chars/4) so CLI runs still get numbers.

- Prompt tokens: 5522
- Completion tokens: 6485
- Reasoning tokens: 0
- Total tokens: 12007
- Cached prompt tokens: 0
- Cache write tokens: 0
- Reported cost: 0
- Upstream inference cost: 0
- Attempts with token counts: 76 (provider 0, estimated 76)
- Answer bytes (sum): 25930
- Attempt duration avg/min/max: 8596 / 5551 / 18770 ms

## Groups (single-sample result + mean score)

| Fixture | Runner | K | Passes | Passed | Mean | Best |
| --- | --- | ---: | ---: | --- | ---: | --- |
| gmod.ents-iterator.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.ents-iterator.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.hook-add.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.hook-add.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.isvalid.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.isvalid.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.net-receivers.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.net-receivers.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.angle-zero.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | 0.50 | partial |
| gmod.perf.angle-zero.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | 0.00 | incorrect |
| gmod.perf.config-var.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | 0.00 | incorrect |
| gmod.perf.config-var.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.darkrpvar.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.darkrpvar.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.disttosqr.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.disttosqr.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.ents-iterator.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.ents-iterator.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.find-ents-near.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.find-ents-near.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.find-players-near.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.find-players-near.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.for-vs-while.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.for-vs-while.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.hook-once.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.hook-once.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.hudpaint-cache.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.hudpaint-cache.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.local-color.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.local-color.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.local-vs-global.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.local-vs-global.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.localplayer-cache.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.localplayer-cache.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.looking-at.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.looking-at.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | 0.50 | partial |
| gmod.perf.meta-vs-arg.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.meta-vs-arg.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.msg-vs-print.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.msg-vs-print.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.pairs-ipairs-for.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.pairs-ipairs-for.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | 0.00 | incorrect |
| gmod.perf.plain-lua-vs-fn.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.plain-lua-vs-fn.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.player-cache-hooks.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.player-cache-hooks.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.ply-table-index.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.ply-table-index.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | 0.00 | incorrect |
| gmod.perf.seq-tables.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.seq-tables.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.setdrawcolor-split.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.setdrawcolor-split.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.sqlite-vs-mysql.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.sqlite-vs-mysql.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.steamid-cache.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.steamid-cache.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.string-table-concat.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.string-table-concat.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.surface-drawrect.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.surface-drawrect.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.surface-text.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.surface-text.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.table-count-hash.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.table-count-hash.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.table-empty.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.table-empty.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.table-hasvalue.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.table-hasvalue.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | 0.00 | incorrect |
| gmod.perf.table-literal.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.table-literal.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.table-random.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.table-random.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.perf.x-times-x.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.perf.x-times-x.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 1 | yes | 1.00 | pass |
| gmod.player-iterator.v1 | agy/Claude Opus 4.6 (Thinking) | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | agy/Claude Sonnet 4.6 (Thinking) | 1 | 0 | no | 0.50 | partial |

## Attempts

| Fixture | Adapter | Model | # | Status | ms | P/C tok | Reason tok | Src | Bytes | Body | Detail |
| --- | --- | --- | ---: | --- | ---: | ---: | ---: | --- | ---: | --- | --- |
| gmod.ents-iterator.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 9771 | 72/70 | — | estimated | 280 | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 11041 | 78/66 | — | estimated | 261 | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 7659 | 74/80 | — | estimated | 323 | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 8304 | 64/107 | — | estimated | 427 | yes | Matched a pass pattern. |
| gmod.perf.angle-zero.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | incorrect | 18770 | 71/147 | — | estimated | 587 | yes | Did not match a fixture answer pattern. |
| gmod.perf.config-var.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 11423 | 82/123 | — | estimated | 494 | yes | Matched a pass pattern. |
| gmod.perf.darkrpvar.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 7683 | 66/90 | — | estimated | 360 | yes | Matched a pass pattern. |
| gmod.perf.disttosqr.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 9798 | 71/78 | — | estimated | 312 | yes | Matched a pass pattern. |
| gmod.perf.ents-iterator.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 8399 | 66/84 | — | estimated | 336 | yes | Matched a pass pattern. |
| gmod.perf.find-ents-near.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | protocol_error | 11911 | 83/163 | — | estimated | 649 | yes | Answer must contain 0 through 0 candidate loops. |
| gmod.perf.find-players-near.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 13129 | 79/211 | — | estimated | 848 | yes | Matched a pass pattern. |
| gmod.perf.for-vs-while.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 9271 | 65/120 | — | estimated | 478 | yes | Matched a pass pattern. |
| gmod.perf.hook-once.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 11002 | 84/188 | — | estimated | 752 | yes | Matched a pass pattern. |
| gmod.perf.hudpaint-cache.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 12591 | 76/163 | — | estimated | 656 | yes | Matched a pass pattern. |
| gmod.perf.local-color.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 8810 | 63/99 | — | estimated | 394 | yes | Matched a pass pattern. |
| gmod.perf.local-vs-global.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 7055 | 62/89 | — | estimated | 353 | yes | Matched a pass pattern. |
| gmod.perf.localplayer-cache.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 11091 | 70/175 | — | estimated | 700 | yes | Matched a pass pattern. |
| gmod.perf.looking-at.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | partial | 13833 | 91/312 | — | estimated | 1255 | yes | Matched a partial-answer pattern. |
| gmod.perf.meta-vs-arg.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 8022 | 71/91 | — | estimated | 365 | yes | Matched a pass pattern. |
| gmod.perf.msg-vs-print.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 6516 | 63/86 | — | estimated | 344 | yes | Matched a pass pattern. |
| gmod.perf.pairs-ipairs-for.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | incorrect | 9315 | 73/137 | — | estimated | 550 | yes | Did not match a fixture answer pattern. |
| gmod.perf.plain-lua-vs-fn.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 13488 | 76/267 | — | estimated | 1068 | yes | Matched a pass pattern. |
| gmod.perf.player-cache-hooks.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 11933 | 88/281 | — | estimated | 1129 | yes | Matched a pass pattern. |
| gmod.perf.ply-table-index.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | incorrect | 12676 | 69/274 | — | estimated | 1098 | yes | Did not match a fixture answer pattern. |
| gmod.perf.seq-tables.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | protocol_error | 9045 | 79/170 | — | estimated | 680 | yes | Answer must contain 0 through 1 candidate loops. |
| gmod.perf.setdrawcolor-split.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 6611 | 71/63 | — | estimated | 249 | yes | Matched a pass pattern. |
| gmod.perf.sqlite-vs-mysql.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 7703 | 100/127 | — | estimated | 505 | yes | Matched a pass pattern. |
| gmod.perf.steamid-cache.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 12244 | 63/331 | — | estimated | 1326 | yes | Matched a pass pattern. |
| gmod.perf.string-table-concat.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | protocol_error | 8722 | 70/161 | — | estimated | 644 | yes | Answer must contain 0 through 0 candidate loops. |
| gmod.perf.surface-drawrect.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 10049 | 69/117 | — | estimated | 470 | yes | Matched a pass pattern. |
| gmod.perf.surface-text.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 12592 | 76/209 | — | estimated | 835 | yes | Matched a pass pattern. |
| gmod.perf.table-count-hash.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 7020 | 62/90 | — | estimated | 360 | yes | Matched a pass pattern. |
| gmod.perf.table-empty.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 9089 | 73/85 | — | estimated | 341 | yes | Matched a pass pattern. |
| gmod.perf.table-hasvalue.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | incorrect | 9285 | 67/156 | — | estimated | 625 | yes | Matched an incorrect-answer pattern. |
| gmod.perf.table-literal.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 7305 | 70/93 | — | estimated | 372 | yes | Matched a pass pattern. |
| gmod.perf.table-random.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 7987 | 66/67 | — | estimated | 267 | yes | Matched a pass pattern. |
| gmod.perf.x-times-x.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | pass | 7644 | 58/59 | — | estimated | 236 | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | agy | Claude Sonnet 4.6 (Thinking) | 1 | partial | 6862 | 80/83 | — | estimated | 329 | yes | Uses ipairs but still allocates via player.GetAll() each time. |
| gmod.ents-iterator.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 7671 | 72/82 | — | estimated | 326 | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 6689 | 78/56 | — | estimated | 221 | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 7638 | 74/79 | — | estimated | 316 | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 8248 | 64/76 | — | estimated | 302 | yes | Matched a pass pattern. |
| gmod.perf.angle-zero.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | partial | 12722 | 71/98 | — | estimated | 392 | yes | Matched a partial-answer pattern. |
| gmod.perf.config-var.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | incorrect | 8648 | 82/104 | — | estimated | 416 | yes | Matched an incorrect-answer pattern. |
| gmod.perf.darkrpvar.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 6847 | 66/65 | — | estimated | 257 | yes | Matched a pass pattern. |
| gmod.perf.disttosqr.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 8575 | 71/53 | — | estimated | 210 | yes | Matched a pass pattern. |
| gmod.perf.ents-iterator.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 7878 | 66/59 | — | estimated | 235 | yes | Matched a pass pattern. |
| gmod.perf.find-ents-near.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 11500 | 83/102 | — | estimated | 405 | yes | Answer must contain 0 through 0 candidate loops. |
| gmod.perf.find-players-near.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 11059 | 79/147 | — | estimated | 585 | yes | Matched a pass pattern. |
| gmod.perf.for-vs-while.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 10232 | 65/85 | — | estimated | 339 | yes | Matched a pass pattern. |
| gmod.perf.hook-once.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | pass | 9299 | 84/167 | — | estimated | 668 | yes | Matched a pass pattern. |
| gmod.perf.hudpaint-cache.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6898 | 76/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.local-color.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6287 | 63/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.local-vs-global.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6741 | 62/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.localplayer-cache.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6729 | 70/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.looking-at.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6450 | 91/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.meta-vs-arg.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 5618 | 71/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.msg-vs-print.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6552 | 63/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.pairs-ipairs-for.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6334 | 73/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.plain-lua-vs-fn.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6829 | 76/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.player-cache-hooks.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6866 | 88/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.ply-table-index.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 5840 | 69/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.seq-tables.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6840 | 79/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.setdrawcolor-split.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6808 | 71/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.sqlite-vs-mysql.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6667 | 100/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.steamid-cache.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 5551 | 63/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.string-table-concat.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6220 | 70/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.surface-drawrect.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6469 | 69/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.surface-text.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6410 | 76/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.table-count-hash.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6611 | 62/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.table-empty.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6641 | 73/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.table-hasvalue.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6690 | 67/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.table-literal.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6666 | 70/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.table-random.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 6611 | 66/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.perf.x-times-x.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 5641 | 58/0 | — | estimated | — | no | CLI exited without a valid strict response. |
| gmod.player-iterator.v1 | agy | Claude Opus 4.6 (Thinking) | 1 | protocol_error | 5639 | 80/0 | — | estimated | — | no | CLI exited without a valid strict response. |

## Files (audit pack)

Same run directory contains:

- `run.json` — full structured artifact (answers + per-attempt metrics; raw stdout stripped)
- `metadata.json` — run provenance + `usageTotals` (tokens, cost, timing)
- `leaderboard.json` / `leaderboard.csv` — fixture-normalized model ranking, diagnostics, tokens, and timing
- `attempts.jsonl` / `attempts.csv` — one row per attempt + tokens/timing/sizes
- `preds.jsonl` — SWE-style predictions with usage fields
- `responses/<adapter>/<model>/<fixture>--kN.txt` — graded answer files
- `raw/*.log` — redacted stdout/stderr when keepRaw is on (default)

Compare two models: `bun run bench compare --run <run.json> --model "A" --model "B"`