Run 9e2fc537
Jul 10, 2026, 12:56
OpenRouter · 31 models · repeat ×1
9e2fc537-7c24-44e1-8e0b-5f59466a461e
Downloads
Audit pack
Results
31 models
| # | Adapter | Model | Think | Pass % | Cover % | Score | Outcomes | Time |
|---|---|---|---|---|---|---|---|---|
| 1 | OpenRouter | nemotron-3-ultra-550b-a55b | High | 80.0% | 100.0% | 90.0% | 4/1/0/0 | 6.3s |
#1 nemotron-3-ultra-550b-a55bHighPass rate80.0% 4 pass · 5 scored Coverage100.0% 5 / 5 evidence-bearing Verified score90.0% 95% interval — · 1 run(s) pass@k80.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 2 | OpenRouter | nemotron-3-ultra-550b-a55b | Medium | 80.0% | 100.0% | 90.0% | 4/1/0/0 | 13.4s |
#2 nemotron-3-ultra-550b-a55bMediumPass rate80.0% 4 pass · 5 scored Coverage100.0% 5 / 5 evidence-bearing Verified score90.0% 95% interval — · 1 run(s) pass@k80.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 3 | OpenRouter | gpt-oss-20b | Low | 80.0% | 100.0% | 90.0% | 4/1/0/0 | 3.3s |
#3 gpt-oss-20bLowPass rate80.0% 4 pass · 5 scored Coverage100.0% 5 / 5 evidence-bearing Verified score90.0% 95% interval — · 1 run(s) pass@k80.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 4 | OpenRouter | laguna-xs-2.1 | — | 80.0% | 100.0% | 90.0% | 4/1/0/0 | 4.8s |
#4 laguna-xs-2.1Pass rate80.0% 4 pass · 5 scored Coverage100.0% 5 / 5 evidence-bearing Verified score90.0% 95% interval — · 1 run(s) pass@k80.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 5 | OpenRouter | hy3 | High | 80.0% | 100.0% | 90.0% | 4/1/0/0 | 20.6s |
#5 hy3HighPass rate80.0% 4 pass · 5 scored Coverage100.0% 5 / 5 evidence-bearing Verified score90.0% 95% interval — · 1 run(s) pass@k80.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 6 | OpenRouter | hy3 | Low | 80.0% | 100.0% | 90.0% | 4/1/0/0 | 12.7s |
#6 hy3LowPass rate80.0% 4 pass · 5 scored Coverage100.0% 5 / 5 evidence-bearing Verified score90.0% 95% interval — · 1 run(s) pass@k80.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 7 | OpenRouter | hy3 | None | 80.0% | 100.0% | 90.0% | 4/1/0/0 | 2.6s |
#7 hy3NonePass rate80.0% 4 pass · 5 scored Coverage100.0% 5 / 5 evidence-bearing Verified score90.0% 95% interval — · 1 run(s) pass@k80.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 8 | OpenRouter | nemotron-3-nano-30b-a3b | — | 100.0% | 80.0% | 80.0% | 4/0/0/1 | 3.7s |
#8 nemotron-3-nano-30b-a3bPass rate100.0% 4 pass · 4 scored Coverage80.0% 4 / 5 evidence-bearing Verified score80.0% 95% interval — · 1 run(s) pass@k100.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 9 | OpenRouter | nemotron-nano-12b-v2-vl | — | 100.0% | 80.0% | 80.0% | 4/0/0/1 | 13.8s |
#9 nemotron-nano-12b-v2-vlPass rate100.0% 4 pass · 4 scored Coverage80.0% 4 / 5 evidence-bearing Verified score80.0% 95% interval — · 1 run(s) pass@k100.0% 4/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures4/5 passed in linked runplayer-iteratorents-iteratorhook-addisvalidnet-receivers | ||||||||
| 10 | OpenRouter | gpt-oss-20b | Medium | 75.0% | 80.0% | 70.0% | 3/1/0/1 | 9.3s |
#10 gpt-oss-20bMediumPass rate75.0% 3 pass · 4 scored Coverage80.0% 4 / 5 evidence-bearing Verified score70.0% 95% interval — · 1 run(s) pass@k75.0% 3/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures3/5 passed in linked runents-iteratorplayer-iteratorhook-addisvalidnet-receivers | ||||||||
| 11 | OpenRouter | north-mini-code | — | 100.0% | 60.0% | 60.0% | 3/0/0/2 | 9.7s |
#11 north-mini-codePass rate100.0% 3 pass · 3 scored Coverage60.0% 3 / 5 evidence-bearing Verified score60.0% 95% interval — · 1 run(s) pass@k100.0% 3/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures3/5 passed in linked runents-iteratorplayer-iteratorhook-addisvalidnet-receivers | ||||||||
| 12 | OpenRouter | nemotron-3-super-120b-a12b | Medium | 100.0% | 60.0% | 60.0% | 3/0/0/2 | 11.5s |
#12 nemotron-3-super-120b-a12bMediumPass rate100.0% 3 pass · 3 scored Coverage60.0% 3 / 5 evidence-bearing Verified score60.0% 95% interval — · 1 run(s) pass@k100.0% 3/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures3/5 passed in linked runents-iteratorplayer-iteratorhook-addisvalidnet-receivers | ||||||||
| 13 | OpenRouter | nemotron-3-super-120b-a12b | Low | 66.7% | 60.0% | 50.0% | 2/1/0/2 | 14.3s |
#13 nemotron-3-super-120b-a12bLowPass rate66.7% 2 pass · 3 scored Coverage60.0% 3 / 5 evidence-bearing Verified score50.0% 95% interval — · 1 run(s) pass@k66.7% 2/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures2/5 passed in linked runents-iteratorhook-addplayer-iteratorisvalidnet-receivers | ||||||||
| 14 | OpenRouter | nemotron-3-nano-omni-30b-a3b-reasoning | — | 0.0% | 0.0% | 0.0% | 0/0/0/0 | 120.0s |
#14 nemotron-3-nano-omni-30b-a3b-reasoningPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 15 | OpenRouter | dolphin-mistral-24b-venice-edition | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 1.0s |
#15 dolphin-mistral-24b-venice-editionPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 16 | OpenRouter | gemma-4-26b-a4b-it | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 511ms |
#16 gemma-4-26b-a4b-itPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 17 | OpenRouter | gemma-4-31b-it | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 1.3s |
#17 gemma-4-31b-itPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 18 | OpenRouter | lfm-2.5-1.2b-instruct | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 324ms |
#18 lfm-2.5-1.2b-instructPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 19 | OpenRouter | lfm-2.5-1.2b-thinking | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 270ms |
#19 lfm-2.5-1.2b-thinkingPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 20 | OpenRouter | llama-3.2-3b-instruct | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 837ms |
#20 llama-3.2-3b-instructPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 21 | OpenRouter | llama-3.3-70b-instruct | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 922ms |
#21 llama-3.3-70b-instructPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 22 | OpenRouter | hermes-3-llama-3.1-405b | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 1.0s |
#22 hermes-3-llama-3.1-405bPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 23 | OpenRouter | nemotron-3.5-content-safety | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 3.3s |
#23 nemotron-3.5-content-safetyPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 24 | OpenRouter | nemotron-nano-9b-v2 | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 10.7s |
#24 nemotron-nano-9b-v2Pass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 25 | OpenRouter | gpt-oss-120b | High | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 2.4s |
#25 gpt-oss-120bHighPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 26 | OpenRouter | gpt-oss-120b | Low | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 2.5s |
#26 gpt-oss-120bLowPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 27 | OpenRouter | gpt-oss-120b | Medium | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 2.4s |
#27 gpt-oss-120bMediumPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 28 | OpenRouter | gpt-oss-20b | High | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 10.8s |
#28 gpt-oss-20bHighPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 29 | OpenRouter | laguna-m.1 | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 15.8s |
#29 laguna-m.1Pass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 30 | OpenRouter | qwen3-coder | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 883ms |
#30 qwen3-coderPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
| 31 | OpenRouter | qwen3-next-80b-a3b-instruct | — | 0.0% | 0.0% | 0.0% | 0/0/0/5 | 941ms |
#31 qwen3-next-80b-a3b-instructPass rate0.0% 0 pass · 0 scored Coverage0.0% 0 / 5 evidence-bearing Verified score0.0% 95% interval — · 1 run(s) pass@k0.0% 0/5 fixtures ever solved Outcome mix
Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel
Run facts
Latest run fixtures0/5 passed in linked runents-iteratorhook-addisvalidnet-receiversplayer-iterator | ||||||||
Verified score aggregates compatible runs. Outcomes = pass / partial / wrong / invalid output; harness failures are excluded. Click a row for evidence and the linked run's fixtures.
Fixtures
5
gmod.ents-iterator.v1gmod.hook-add.v1gmod.isvalid.v1gmod.net-receivers.v1gmod.player-iterator.v1