# GMod Bench report

Run: `5aadc831-52d7-4546-9df4-67fafa3b1151`
Fixtures: `gmod.player-iterator.v1`
Repeat: 1 · Concurrency: 2
Started: 2026-07-10T17:50:42.075Z · Completed: 2026-07-10T17:51:09.382Z
Wall clock: 27s · Sum of attempt durations: 27279ms (avg 27279ms, min 27279ms, max 27279ms)

## Model leaderboard

Sorted by **coverage** (scorable attempts / all attempts), then pass count, repeat consistency, and quality.

| Rank | Runner / model | Pass | Partial | Incorrect | Scored | Coverage | Pass rate | Fixture pass | Quality | Protocol err | Fixtures | Avg ms | Prompt/Compl tok |
| ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | `opencode/opencode/big-pickle` | 1 | 0 | 0 | 1 | 100.0% | 100.0% | 100.0% | 1.00 | 0 | 1/1 | 27279 | 21083/52 |

## Rates

- Scored attempts: 1
- Pass / partial / incorrect: 1 / 0 / 0
- Pass rate (scored): 1/1
- Fixture pass rate (passed fixture groups / scored fixture groups): 1/1
- Overall mean score (pass=1, partial=0.5): 1.000
- Protocol errors: 0
- Policy violations: 0
- Trace errors: 0
- Timeouts: 0
- Unavailable: 0
- Unsupported: 0

## Usage & timing

Token counts: **provider** when the runner reports them (OpenRouter); otherwise **estimated** (~chars/4) so CLI runs still get numbers.

- Prompt tokens: 21083
- Completion tokens: 52
- Reasoning tokens: 177
- Total tokens: 22336
- Cached prompt tokens: 1024
- Cache write tokens: 0
- Reported cost: 0
- Upstream inference cost: 0
- Attempts with token counts: 1 (provider 1, estimated 0)
- Answer bytes (sum): 209
- Attempt duration avg/min/max: 27279 / 27279 / 27279 ms

## Groups (single-sample result + mean score)

| Fixture | Runner | K | Passes | Passed | Mean | Best |
| --- | --- | ---: | ---: | --- | ---: | --- |
| gmod.player-iterator.v1 | opencode/opencode/big-pickle | 1 | 1 | yes | 1.00 | pass |

## Attempts

| Fixture | Adapter | Model | # | Status | ms | P/C tok | Reason tok | Src | Bytes | Body | Detail |
| --- | --- | --- | ---: | --- | ---: | ---: | ---: | --- | ---: | --- | --- |
| gmod.player-iterator.v1 | opencode | opencode/big-pickle | 1 | pass | 27279 | 21083/52 | 177 | provider | 209 | yes | Uses player.Iterator() (wiki-recommended Lua-cached iterator). |

## Files (audit pack)

Same run directory contains:

- `run.json` — full structured artifact (answers + per-attempt metrics; raw stdout stripped)
- `metadata.json` — run provenance + `usageTotals` (tokens, cost, timing)
- `leaderboard.json` / `leaderboard.csv` — model ranking (coverage-first, tokens, ms)
- `attempts.jsonl` / `attempts.csv` — one row per attempt + tokens/timing/sizes
- `preds.jsonl` — SWE-style predictions with usage fields
- `responses/<adapter>/<model>/<fixture>--kN.txt` — graded answer files
- `raw/*.log` — redacted stdout/stderr when keepRaw is on (default)

Compare two models: `bun run bench compare --run <run.json> --model "A" --model "B"`