# GMod Bench report

Run: `9e2fc537-7c24-44e1-8e0b-5f59466a461e`
Fixtures: `gmod.ents-iterator.v1`, `gmod.hook-add.v1`, `gmod.isvalid.v1`, `gmod.net-receivers.v1`, `gmod.player-iterator.v1`
Repeat: 1 · Concurrency: 2
Started: 2026-07-10T12:38:58.286Z · Completed: 2026-07-10T12:56:01.492Z
Wall clock: 1023s · Sum of attempt durations: 1509102ms (avg 9736ms, min 138ms, max 120062ms)

## Model leaderboard

Sorted by **fixture score** (each fixture weighted equally; failures score zero), then fixture solve rate, coverage, and pass rate.

| Rank | Runner / model | Fixture score | Fixture solve | Pass | Partial | Incorrect | Scored | Coverage | Pass rate | Fixture pass | Scored quality | Protocol err | Fixtures | Avg ms | Prompt/Compl tok |
| ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | `openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@high` | 90.0% | 80.0% | 4 | 1 | 0 | 5 | 100.0% | 80.0% | 80.0% | 0.90 | 0 | 4/5 | 6296 | 934/3667 |
| 2 | `openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@medium` | 90.0% | 80.0% | 4 | 1 | 0 | 5 | 100.0% | 80.0% | 80.0% | 0.90 | 0 | 4/5 | 13417 | 934/6152 |
| 3 | `openrouter/openai/gpt-oss-20b:free@low` | 90.0% | 80.0% | 4 | 1 | 0 | 5 | 100.0% | 80.0% | 80.0% | 0.90 | 0 | 4/5 | 3268 | 1222/583 |
| 4 | `openrouter/poolside/laguna-xs-2.1:free` | 90.0% | 80.0% | 4 | 1 | 0 | 5 | 100.0% | 80.0% | 80.0% | 0.90 | 0 | 4/5 | 4759 | 909/305 |
| 5 | `openrouter/tencent/hy3:free@high` | 90.0% | 80.0% | 4 | 1 | 0 | 5 | 100.0% | 80.0% | 80.0% | 0.90 | 0 | 4/5 | 20619 | 904/5913 |
| 6 | `openrouter/tencent/hy3:free@low` | 90.0% | 80.0% | 4 | 1 | 0 | 5 | 100.0% | 80.0% | 80.0% | 0.90 | 0 | 4/5 | 12696 | 904/3559 |
| 7 | `openrouter/tencent/hy3:free@none` | 90.0% | 80.0% | 4 | 1 | 0 | 5 | 100.0% | 80.0% | 80.0% | 0.90 | 0 | 4/5 | 2568 | 919/282 |
| 8 | `openrouter/nvidia/nemotron-3-nano-30b-a3b:free` | 80.0% | 80.0% | 4 | 0 | 0 | 4 | 80.0% | 100.0% | 100.0% | 1.00 | 1 | 4/5 | 3680 | 749/2103 |
| 9 | `openrouter/nvidia/nemotron-nano-12b-v2-vl:free` | 80.0% | 80.0% | 4 | 0 | 0 | 4 | 80.0% | 100.0% | 100.0% | 1.00 | 1 | 4/5 | 13769 | 909/2860 |
| 10 | `openrouter/openai/gpt-oss-20b:free@medium` | 70.0% | 60.0% | 3 | 1 | 0 | 4 | 80.0% | 75.0% | 75.0% | 0.88 | 1 | 3/5 | 9266 | 1216/1363 |
| 11 | `openrouter/cohere/north-mini-code:free` | 60.0% | 60.0% | 3 | 0 | 0 | 3 | 60.0% | 100.0% | 100.0% | 1.00 | 2 | 3/5 | 9673 | 819/3565 |
| 12 | `openrouter/nvidia/nemotron-3-super-120b-a12b:free@medium` | 60.0% | 60.0% | 3 | 0 | 0 | 3 | 60.0% | 100.0% | 100.0% | 1.00 | 2 | 3/5 | 11499 | 749/1253 |
| 13 | `openrouter/nvidia/nemotron-3-super-120b-a12b:free@low` | 50.0% | 40.0% | 2 | 1 | 0 | 3 | 60.0% | 66.7% | 66.7% | 0.83 | 2 | 2/5 | 14342 | 934/1753 |
| 14 | `openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 0 | 0/5 | 120014 | 0/0 |
| 15 | `openrouter/cognitivecomputations/dolphin-mistral-24b-venice-edition:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 1040 | 0/0 |
| 16 | `openrouter/google/gemma-4-26b-a4b-it:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 511 | 0/0 |
| 17 | `openrouter/google/gemma-4-31b-it:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 1300 | 0/0 |
| 18 | `openrouter/liquid/lfm-2.5-1.2b-instruct:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 324 | 0/0 |
| 19 | `openrouter/liquid/lfm-2.5-1.2b-thinking:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 270 | 0/0 |
| 20 | `openrouter/meta-llama/llama-3.2-3b-instruct:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 837 | 0/0 |
| 21 | `openrouter/meta-llama/llama-3.3-70b-instruct:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 922 | 0/0 |
| 22 | `openrouter/nousresearch/hermes-3-llama-3.1-405b:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 1042 | 0/0 |
| 23 | `openrouter/nvidia/nemotron-3.5-content-safety:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 3252 | 2650/1272 |
| 24 | `openrouter/nvidia/nemotron-nano-9b-v2:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 10725 | 909/2041 |
| 25 | `openrouter/openai/gpt-oss-120b:free@high` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 2413 | 0/0 |
| 26 | `openrouter/openai/gpt-oss-120b:free@low` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 2463 | 0/0 |
| 27 | `openrouter/openai/gpt-oss-120b:free@medium` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 2396 | 0/0 |
| 28 | `openrouter/openai/gpt-oss-20b:free@high` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 10837 | 0/0 |
| 29 | `openrouter/poolside/laguna-m.1:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 15795 | 954/2147 |
| 30 | `openrouter/qwen/qwen3-coder:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 883 | 0/0 |
| 31 | `openrouter/qwen/qwen3-next-80b-a3b-instruct:free` | 0.0% | 0.0% | 0 | 0 | 0 | 0 | 0.0% | 0.0% | 0.0% | 0.00 | 5 | 0/5 | 941 | 0/0 |

## Rates

- Scored attempts: 56
- Pass / partial / incorrect: 47 / 9 / 0
- Pass rate (scored): 47/56
- Fixture pass rate (passed fixture groups / scored fixture groups): 47/56
- Overall mean score (pass=1, partial=0.5): —
- Protocol errors: 94
- Policy violations: 0
- Trace errors: 0
- Timeouts: 5
- Unavailable: 0
- Unsupported: 0

## Usage & timing

Token counts: **provider** when the runner reports them (OpenRouter); otherwise **estimated** (~chars/4) so CLI runs still get numbers.

- Prompt tokens: 16615
- Completion tokens: 38818
- Reasoning tokens: 0
- Total tokens: 55433
- Cached prompt tokens: 144
- Cache write tokens: 0
- Reported cost: 0
- Upstream inference cost: 0
- Attempts with token counts: 78 (provider 78, estimated 0)
- Answer bytes (sum): 0
- Attempt duration avg/min/max: 9736 / 138 / 120062 ms

## Groups (single-sample result + mean score)

| Fixture | Runner | K | Passes | Passed | Mean | Best |
| --- | --- | ---: | ---: | --- | ---: | --- |
| gmod.ents-iterator.v1 | openrouter/cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/cohere/north-mini-code:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/google/gemma-4-26b-a4b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/google/gemma-4-31b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/liquid/lfm-2.5-1.2b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/liquid/lfm-2.5-1.2b-thinking:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/meta-llama/llama-3.2-3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/meta-llama/llama-3.3-70b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/nousresearch/hermes-3-llama-3.1-405b:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-3-nano-30b-a3b:free | 1 | 1 | yes | — | pass |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | 0 | no | — | timeout |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@low | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | 1 | yes | — | pass |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | 1 | yes | — | pass |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-3.5-content-safety:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-nano-12b-v2-vl:free | 1 | 1 | yes | — | pass |
| gmod.ents-iterator.v1 | openrouter/nvidia/nemotron-nano-9b-v2:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/openai/gpt-oss-120b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/openai/gpt-oss-120b:free@low | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/openai/gpt-oss-120b:free@medium | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/openai/gpt-oss-20b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/openai/gpt-oss-20b:free@low | 1 | 1 | yes | — | pass |
| gmod.ents-iterator.v1 | openrouter/openai/gpt-oss-20b:free@medium | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/poolside/laguna-m.1:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/poolside/laguna-xs-2.1:free | 1 | 1 | yes | — | pass |
| gmod.ents-iterator.v1 | openrouter/qwen/qwen3-coder:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/qwen/qwen3-next-80b-a3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.ents-iterator.v1 | openrouter/tencent/hy3:free@high | 1 | 1 | yes | — | pass |
| gmod.ents-iterator.v1 | openrouter/tencent/hy3:free@low | 1 | 1 | yes | — | pass |
| gmod.ents-iterator.v1 | openrouter/tencent/hy3:free@none | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/cohere/north-mini-code:free | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/google/gemma-4-26b-a4b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/google/gemma-4-31b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/liquid/lfm-2.5-1.2b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/liquid/lfm-2.5-1.2b-thinking:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/meta-llama/llama-3.2-3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/meta-llama/llama-3.3-70b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/nousresearch/hermes-3-llama-3.1-405b:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-3-nano-30b-a3b:free | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | 0 | no | — | timeout |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@low | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-3.5-content-safety:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-nano-12b-v2-vl:free | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/nvidia/nemotron-nano-9b-v2:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/openai/gpt-oss-120b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/openai/gpt-oss-120b:free@low | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/openai/gpt-oss-120b:free@medium | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/openai/gpt-oss-20b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/openai/gpt-oss-20b:free@low | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/openai/gpt-oss-20b:free@medium | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/poolside/laguna-m.1:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/poolside/laguna-xs-2.1:free | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/qwen/qwen3-coder:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/qwen/qwen3-next-80b-a3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.hook-add.v1 | openrouter/tencent/hy3:free@high | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/tencent/hy3:free@low | 1 | 1 | yes | — | pass |
| gmod.hook-add.v1 | openrouter/tencent/hy3:free@none | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/cohere/north-mini-code:free | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/google/gemma-4-26b-a4b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/google/gemma-4-31b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/liquid/lfm-2.5-1.2b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/liquid/lfm-2.5-1.2b-thinking:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/meta-llama/llama-3.2-3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/meta-llama/llama-3.3-70b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/nousresearch/hermes-3-llama-3.1-405b:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-3-nano-30b-a3b:free | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | 0 | no | — | timeout |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@low | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-3.5-content-safety:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-nano-12b-v2-vl:free | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/nvidia/nemotron-nano-9b-v2:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/openai/gpt-oss-120b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/openai/gpt-oss-120b:free@low | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/openai/gpt-oss-120b:free@medium | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/openai/gpt-oss-20b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/openai/gpt-oss-20b:free@low | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/openai/gpt-oss-20b:free@medium | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/poolside/laguna-m.1:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/poolside/laguna-xs-2.1:free | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/qwen/qwen3-coder:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/qwen/qwen3-next-80b-a3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.isvalid.v1 | openrouter/tencent/hy3:free@high | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/tencent/hy3:free@low | 1 | 1 | yes | — | pass |
| gmod.isvalid.v1 | openrouter/tencent/hy3:free@none | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/cohere/north-mini-code:free | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/google/gemma-4-26b-a4b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/google/gemma-4-31b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/liquid/lfm-2.5-1.2b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/liquid/lfm-2.5-1.2b-thinking:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/meta-llama/llama-3.2-3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/meta-llama/llama-3.3-70b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/nousresearch/hermes-3-llama-3.1-405b:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-3-nano-30b-a3b:free | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | 0 | no | — | timeout |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@low | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-3.5-content-safety:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-nano-12b-v2-vl:free | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/nvidia/nemotron-nano-9b-v2:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/openai/gpt-oss-120b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/openai/gpt-oss-120b:free@low | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/openai/gpt-oss-120b:free@medium | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/openai/gpt-oss-20b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/openai/gpt-oss-20b:free@low | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/openai/gpt-oss-20b:free@medium | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/poolside/laguna-m.1:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/poolside/laguna-xs-2.1:free | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/qwen/qwen3-coder:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/qwen/qwen3-next-80b-a3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.net-receivers.v1 | openrouter/tencent/hy3:free@high | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/tencent/hy3:free@low | 1 | 1 | yes | — | pass |
| gmod.net-receivers.v1 | openrouter/tencent/hy3:free@none | 1 | 1 | yes | — | pass |
| gmod.player-iterator.v1 | openrouter/cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/cohere/north-mini-code:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/google/gemma-4-26b-a4b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/google/gemma-4-31b-it:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/liquid/lfm-2.5-1.2b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/liquid/lfm-2.5-1.2b-thinking:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/meta-llama/llama-3.2-3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/meta-llama/llama-3.3-70b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/nousresearch/hermes-3-llama-3.1-405b:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-3-nano-30b-a3b:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | 0 | no | — | timeout |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@low | 1 | 0 | no | — | partial |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | 0 | no | — | partial |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | 0 | no | — | partial |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-3.5-content-safety:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-nano-12b-v2-vl:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/nvidia/nemotron-nano-9b-v2:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/openai/gpt-oss-120b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/openai/gpt-oss-120b:free@low | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/openai/gpt-oss-120b:free@medium | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/openai/gpt-oss-20b:free@high | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/openai/gpt-oss-20b:free@low | 1 | 0 | no | — | partial |
| gmod.player-iterator.v1 | openrouter/openai/gpt-oss-20b:free@medium | 1 | 0 | no | — | partial |
| gmod.player-iterator.v1 | openrouter/poolside/laguna-m.1:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/poolside/laguna-xs-2.1:free | 1 | 0 | no | — | partial |
| gmod.player-iterator.v1 | openrouter/qwen/qwen3-coder:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/qwen/qwen3-next-80b-a3b-instruct:free | 1 | 0 | no | — | protocol_error |
| gmod.player-iterator.v1 | openrouter/tencent/hy3:free@high | 1 | 0 | no | — | partial |
| gmod.player-iterator.v1 | openrouter/tencent/hy3:free@low | 1 | 0 | no | — | partial |
| gmod.player-iterator.v1 | openrouter/tencent/hy3:free@none | 1 | 0 | no | — | partial |

## Attempts

| Fixture | Adapter | Model | # | Status | ms | P/C tok | Reason tok | Src | Bytes | Body | Detail |
| --- | --- | --- | ---: | --- | ---: | ---: | ---: | --- | ---: | --- | --- |
| gmod.ents-iterator.v1 | openrouter | cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | protocol_error | 1222 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | protocol_error | 1441 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | protocol_error | 953 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | protocol_error | 943 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | cognitivecomputations/dolphin-mistral-24b-venice-edition:free | 1 | protocol_error | 643 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | cohere/north-mini-code:free | 1 | protocol_error | 8639 | 160/793 | — | — | — | yes | Answer must contain 1 through 1 candidate loops. |
| gmod.hook-add.v1 | openrouter | cohere/north-mini-code:free | 1 | pass | 7714 | 169/525 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | cohere/north-mini-code:free | 1 | pass | 5454 | 166/676 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | cohere/north-mini-code:free | 1 | pass | 6757 | 161/547 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | cohere/north-mini-code:free | 1 | protocol_error | 19802 | 163/1024 | — | — | — | no | OpenRouter message content was empty or missing. |
| gmod.ents-iterator.v1 | openrouter | google/gemma-4-26b-a4b-it:free | 1 | protocol_error | 1054 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | google/gemma-4-26b-a4b-it:free | 1 | protocol_error | 745 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | google/gemma-4-26b-a4b-it:free | 1 | protocol_error | 269 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | google/gemma-4-26b-a4b-it:free | 1 | protocol_error | 222 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | google/gemma-4-26b-a4b-it:free | 1 | protocol_error | 266 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | google/gemma-4-31b-it:free | 1 | protocol_error | 1810 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | google/gemma-4-31b-it:free | 1 | protocol_error | 1476 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | google/gemma-4-31b-it:free | 1 | protocol_error | 1612 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | google/gemma-4-31b-it:free | 1 | protocol_error | 763 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | google/gemma-4-31b-it:free | 1 | protocol_error | 839 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | liquid/lfm-2.5-1.2b-instruct:free | 1 | protocol_error | 477 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | liquid/lfm-2.5-1.2b-instruct:free | 1 | protocol_error | 143 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | liquid/lfm-2.5-1.2b-instruct:free | 1 | protocol_error | 159 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | liquid/lfm-2.5-1.2b-instruct:free | 1 | protocol_error | 460 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | liquid/lfm-2.5-1.2b-instruct:free | 1 | protocol_error | 381 | —/— | — | — | — | no | OpenRouter HTTP 502. |
| gmod.ents-iterator.v1 | openrouter | liquid/lfm-2.5-1.2b-thinking:free | 1 | protocol_error | 501 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | liquid/lfm-2.5-1.2b-thinking:free | 1 | protocol_error | 206 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | liquid/lfm-2.5-1.2b-thinking:free | 1 | protocol_error | 138 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | liquid/lfm-2.5-1.2b-thinking:free | 1 | protocol_error | 148 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | liquid/lfm-2.5-1.2b-thinking:free | 1 | protocol_error | 358 | —/— | — | — | — | no | OpenRouter HTTP 502. |
| gmod.ents-iterator.v1 | openrouter | meta-llama/llama-3.2-3b-instruct:free | 1 | protocol_error | 1137 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | meta-llama/llama-3.2-3b-instruct:free | 1 | protocol_error | 1016 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | meta-llama/llama-3.2-3b-instruct:free | 1 | protocol_error | 731 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | meta-llama/llama-3.2-3b-instruct:free | 1 | protocol_error | 701 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | meta-llama/llama-3.2-3b-instruct:free | 1 | protocol_error | 600 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | meta-llama/llama-3.3-70b-instruct:free | 1 | protocol_error | 1162 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | meta-llama/llama-3.3-70b-instruct:free | 1 | protocol_error | 1007 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | meta-llama/llama-3.3-70b-instruct:free | 1 | protocol_error | 813 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | meta-llama/llama-3.3-70b-instruct:free | 1 | protocol_error | 947 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | meta-llama/llama-3.3-70b-instruct:free | 1 | protocol_error | 681 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | nousresearch/hermes-3-llama-3.1-405b:free | 1 | protocol_error | 1216 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | nousresearch/hermes-3-llama-3.1-405b:free | 1 | protocol_error | 1180 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | nousresearch/hermes-3-llama-3.1-405b:free | 1 | protocol_error | 1101 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | nousresearch/hermes-3-llama-3.1-405b:free | 1 | protocol_error | 1034 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | nousresearch/hermes-3-llama-3.1-405b:free | 1 | protocol_error | 681 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-3-nano-30b-a3b:free | 1 | pass | 3491 | 183/597 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-3-nano-30b-a3b:free | 1 | pass | 5914 | 191/452 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-3-nano-30b-a3b:free | 1 | pass | 2493 | 189/422 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-3-nano-30b-a3b:free | 1 | pass | 6200 | 186/632 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-3-nano-30b-a3b:free | 1 | protocol_error | 303 | —/— | — | — | — | no | OpenRouter error: Upstream error from Nvidia: ResourceExhausted: Worker local total request limit reached (145/32) |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | timeout | 120003 | —/— | — | — | — | no | HTTP request exceeded the attempt timeout. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | timeout | 120001 | —/— | — | — | — | no | HTTP request exceeded the attempt timeout. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | timeout | 120002 | —/— | — | — | — | no | HTTP request exceeded the attempt timeout. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | timeout | 120003 | —/— | — | — | — | no | HTTP request exceeded the attempt timeout. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | 1 | timeout | 120062 | —/— | — | — | — | no | HTTP request exceeded the attempt timeout. |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | protocol_error | 7816 | 183/474 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | pass | 12158 | 191/211 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | pass | 32658 | 189/270 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | pass | 4488 | 186/298 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@medium | 1 | protocol_error | 375 | —/— | — | — | — | no | OpenRouter error: Upstream error from Nvidia: ResourceExhausted: Worker local total request limit reached (32/32) |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@low | 1 | protocol_error | 5928 | 183/512 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@low | 1 | protocol_error | 28315 | 191/317 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@low | 1 | pass | 7026 | 189/240 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@low | 1 | pass | 15773 | 186/261 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-3-super-120b-a12b:free@low | 1 | partial | 14670 | 185/423 | — | — | — | yes | Still materializes player.GetAll(). |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | pass | 14541 | 183/1357 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | pass | 4583 | 191/249 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | pass | 3901 | 189/466 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | pass | 3380 | 186/412 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@high | 1 | partial | 5077 | 185/1183 | — | — | — | yes | Uses sequential traversal but still materializes player.GetAll(). |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | pass | 41870 | 183/2679 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | pass | 1900 | 191/249 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | pass | 3932 | 189/460 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | pass | 3636 | 186/449 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-3-ultra-550b-a55b:free@medium | 1 | partial | 15745 | 185/2315 | — | — | — | yes | Uses sequential traversal but still materializes player.GetAll(). |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-3.5-content-safety:free | 1 | protocol_error | 3230 | 526/252 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-3.5-content-safety:free | 1 | protocol_error | 3255 | 535/246 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-3.5-content-safety:free | 1 | protocol_error | 3238 | 532/262 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-3.5-content-safety:free | 1 | protocol_error | 3117 | 528/257 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-3.5-content-safety:free | 1 | protocol_error | 3419 | 529/255 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-nano-12b-v2-vl:free | 1 | pass | 18102 | 178/791 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-nano-12b-v2-vl:free | 1 | pass | 6706 | 186/324 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-nano-12b-v2-vl:free | 1 | pass | 6258 | 184/237 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-nano-12b-v2-vl:free | 1 | pass | 11496 | 181/484 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-nano-12b-v2-vl:free | 1 | protocol_error | 26285 | 180/1024 | — | — | — | no | OpenRouter message content was empty or missing. |
| gmod.ents-iterator.v1 | openrouter | nvidia/nemotron-nano-9b-v2:free | 1 | protocol_error | 27957 | 178/873 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.hook-add.v1 | openrouter | nvidia/nemotron-nano-9b-v2:free | 1 | protocol_error | 7237 | 186/244 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.isvalid.v1 | openrouter | nvidia/nemotron-nano-9b-v2:free | 1 | protocol_error | 4101 | 184/223 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.net-receivers.v1 | openrouter | nvidia/nemotron-nano-9b-v2:free | 1 | protocol_error | 7174 | 181/266 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.player-iterator.v1 | openrouter | nvidia/nemotron-nano-9b-v2:free | 1 | protocol_error | 7156 | 180/435 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.ents-iterator.v1 | openrouter | openai/gpt-oss-120b:free@high | 1 | protocol_error | 2263 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | openai/gpt-oss-120b:free@high | 1 | protocol_error | 2902 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | openai/gpt-oss-120b:free@high | 1 | protocol_error | 2599 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | openai/gpt-oss-120b:free@high | 1 | protocol_error | 2157 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | openai/gpt-oss-120b:free@high | 1 | protocol_error | 2145 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | openai/gpt-oss-120b:free@medium | 1 | protocol_error | 2168 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | openai/gpt-oss-120b:free@medium | 1 | protocol_error | 2330 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | openai/gpt-oss-120b:free@medium | 1 | protocol_error | 2538 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | openai/gpt-oss-120b:free@medium | 1 | protocol_error | 2906 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | openai/gpt-oss-120b:free@medium | 1 | protocol_error | 2037 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | openai/gpt-oss-120b:free@low | 1 | protocol_error | 2358 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | openai/gpt-oss-120b:free@low | 1 | protocol_error | 2063 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | openai/gpt-oss-120b:free@low | 1 | protocol_error | 2125 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | openai/gpt-oss-120b:free@low | 1 | protocol_error | 2754 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | openai/gpt-oss-120b:free@low | 1 | protocol_error | 3017 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | openai/gpt-oss-20b:free@high | 1 | protocol_error | 11161 | —/— | — | — | — | no | OpenRouter error: Internal Server Error |
| gmod.hook-add.v1 | openrouter | openai/gpt-oss-20b:free@high | 1 | protocol_error | 10729 | —/— | — | — | — | no | OpenRouter error: Internal Server Error |
| gmod.isvalid.v1 | openrouter | openai/gpt-oss-20b:free@high | 1 | protocol_error | 10821 | —/— | — | — | — | no | OpenRouter error: Internal Server Error |
| gmod.net-receivers.v1 | openrouter | openai/gpt-oss-20b:free@high | 1 | protocol_error | 10739 | —/— | — | — | — | no | OpenRouter error: Internal Server Error |
| gmod.player-iterator.v1 | openrouter | openai/gpt-oss-20b:free@high | 1 | protocol_error | 10736 | —/— | — | — | — | no | OpenRouter error: Internal Server Error |
| gmod.ents-iterator.v1 | openrouter | openai/gpt-oss-20b:free@medium | 1 | protocol_error | 11171 | 241/466 | — | — | — | yes | Answer must contain 1 through 1 candidate loops. |
| gmod.hook-add.v1 | openrouter | openai/gpt-oss-20b:free@medium | 1 | pass | 5339 | 249/181 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | openai/gpt-oss-20b:free@medium | 1 | pass | 2774 | 247/171 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | openai/gpt-oss-20b:free@medium | 1 | pass | 4555 | 242/224 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | openai/gpt-oss-20b:free@medium | 1 | partial | 22492 | 237/321 | — | — | — | yes | Still materializes player.GetAll(). |
| gmod.ents-iterator.v1 | openrouter | openai/gpt-oss-20b:free@low | 1 | pass | 5485 | 241/227 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | openai/gpt-oss-20b:free@low | 1 | pass | 1800 | 249/73 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | openai/gpt-oss-20b:free@low | 1 | pass | 2690 | 247/81 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | openai/gpt-oss-20b:free@low | 1 | pass | 4175 | 242/76 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | openai/gpt-oss-20b:free@low | 1 | partial | 2191 | 243/126 | — | — | — | yes | Still materializes player.GetAll(). |
| gmod.ents-iterator.v1 | openrouter | poolside/laguna-m.1:free | 1 | protocol_error | 27103 | 188/786 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.hook-add.v1 | openrouter | poolside/laguna-m.1:free | 1 | protocol_error | 9151 | 195/274 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.isvalid.v1 | openrouter | poolside/laguna-m.1:free | 1 | protocol_error | 14665 | 194/380 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.net-receivers.v1 | openrouter | poolside/laguna-m.1:free | 1 | protocol_error | 15315 | 188/410 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.player-iterator.v1 | openrouter | poolside/laguna-m.1:free | 1 | protocol_error | 12742 | 189/297 | — | — | — | yes | Answer must contain exactly one fenced code block followed by a reason. |
| gmod.ents-iterator.v1 | openrouter | poolside/laguna-xs-2.1:free | 1 | pass | 674 | 179/57 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | poolside/laguna-xs-2.1:free | 1 | pass | 1186 | 186/62 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | poolside/laguna-xs-2.1:free | 1 | pass | 9012 | 185/58 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | poolside/laguna-xs-2.1:free | 1 | pass | 9859 | 179/61 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | poolside/laguna-xs-2.1:free | 1 | partial | 3064 | 180/67 | — | — | — | yes | Still materializes player.GetAll(). |
| gmod.ents-iterator.v1 | openrouter | qwen/qwen3-coder:free | 1 | protocol_error | 1074 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | qwen/qwen3-coder:free | 1 | protocol_error | 1137 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | qwen/qwen3-coder:free | 1 | protocol_error | 732 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | qwen/qwen3-coder:free | 1 | protocol_error | 591 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | qwen/qwen3-coder:free | 1 | protocol_error | 881 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | qwen/qwen3-next-80b-a3b-instruct:free | 1 | protocol_error | 1246 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.hook-add.v1 | openrouter | qwen/qwen3-next-80b-a3b-instruct:free | 1 | protocol_error | 1007 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.isvalid.v1 | openrouter | qwen/qwen3-next-80b-a3b-instruct:free | 1 | protocol_error | 871 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.net-receivers.v1 | openrouter | qwen/qwen3-next-80b-a3b-instruct:free | 1 | protocol_error | 592 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.player-iterator.v1 | openrouter | qwen/qwen3-next-80b-a3b-instruct:free | 1 | protocol_error | 987 | —/— | — | — | — | no | OpenRouter HTTP 429. |
| gmod.ents-iterator.v1 | openrouter | tencent/hy3:free@high | 1 | pass | 15706 | 177/1690 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | tencent/hy3:free@high | 1 | pass | 11864 | 186/590 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | tencent/hy3:free@high | 1 | pass | 15799 | 183/780 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | tencent/hy3:free@high | 1 | pass | 31294 | 179/1637 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | tencent/hy3:free@high | 1 | partial | 28432 | 179/1216 | — | — | — | yes | Still materializes player.GetAll(). |
| gmod.ents-iterator.v1 | openrouter | tencent/hy3:free@low | 1 | pass | 11468 | 177/1197 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | tencent/hy3:free@low | 1 | pass | 8078 | 186/365 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | tencent/hy3:free@low | 1 | pass | 7120 | 183/346 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | tencent/hy3:free@low | 1 | pass | 12316 | 179/610 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | tencent/hy3:free@low | 1 | partial | 24499 | 179/1041 | — | — | — | yes | Still materializes player.GetAll(). |
| gmod.ents-iterator.v1 | openrouter | tencent/hy3:free@none | 1 | pass | 2040 | 180/57 | — | — | — | yes | Matched a pass pattern. |
| gmod.hook-add.v1 | openrouter | tencent/hy3:free@none | 1 | pass | 2366 | 189/47 | — | — | — | yes | Registers a named hook with hook.Add. |
| gmod.isvalid.v1 | openrouter | tencent/hy3:free@none | 1 | pass | 2319 | 186/47 | — | — | — | yes | Uses IsValid for entity/object validity. |
| gmod.net-receivers.v1 | openrouter | tencent/hy3:free@none | 1 | pass | 2617 | 182/68 | — | — | — | yes | Matched a pass pattern. |
| gmod.player-iterator.v1 | openrouter | tencent/hy3:free@none | 1 | partial | 3498 | 182/63 | — | — | — | yes | Still materializes player.GetAll(). |

## Files (audit pack)

Same run directory contains:

- `run.json` — full structured artifact (answers + per-attempt metrics; raw stdout stripped)
- `metadata.json` — run provenance + `usageTotals` (tokens, cost, timing)
- `leaderboard.json` / `leaderboard.csv` — fixture-normalized model ranking, diagnostics, tokens, and timing
- `attempts.jsonl` / `attempts.csv` — one row per attempt + tokens/timing/sizes
- `preds.jsonl` — SWE-style predictions with usage fields
- `responses/<adapter>/<model>/<fixture>--kN.txt` — graded answer files
- `raw/*.log` — redacted stdout/stderr when keepRaw is on (default)

Compare two models: `bun run bench compare --run <run.json> --model "A" --model "B"`