Leaderboard

Jul 11, 2026

Find the strongest models for GMod coding within the broadest comparable fixture cohort. Other published runs remain available in the archive.

Fixtures

Model rows

Runs merged

Adapters

AntiGravityCodexOpenCodeOpenRouter

#	Adapter	Model	Think	Pass %	Cover %	Score	Outcomes	Time
1	Codex	gpt-5.6-terra ddc5aa24	Max	94.7%	100.0%	96.7%	72/3/1/0	21.6s
#1 gpt-5.6-terra Max Codexcodex-cliRun ddc5aa24 Open run Pass rate94.7% 72 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score96.7% 95% interval 85.5–99.3% · 2 run(s) Fixtures37/38 fixtures passed / attempted Outcome mix Pass72 Partial3 Wrong1 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed72 Fixtures solved37/38 Run facts Avg time 21.6s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 85.5–99.3% Suite codex-cli Raw label codex/gpt-5.6-terra@max Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-hudpaint-cacheperf-find-ents-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
2	AntiGravity	Gemini 3.5 Flash c99506be	High	92.1%	100.0%	96.1%	105/9/0/0	6.9s
#2 Gemini 3.5 Flash High AntiGravityagyRun c99506be Open run Pass rate92.1% 105 pass · 114 scored Coverage100.0% 114 / 114 evidence-bearing Verified score96.1% 95% interval 84.6–99.1% · 3 run(s) Fixtures37/38 fixtures passed / attempted Outcome mix Pass105 Partial9 Wrong0 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored114 Passed105 Fixtures solved37/38 Run facts Avg time 6.9s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 84.6–99.1% Suite agy Raw label agy/Gemini 3.5 Flash (High) Run id c99506be-e7cc-4900-90b0-2a059597aa89 Latest run fixtures 32/38 passed in linked run perf-string-table-concatperf-config-varperf-seq-tablesperf-localplayer-cacheperf-looking-atperf-steamid-cacheents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
3	Codex	gpt-5.6-luna ddc5aa24	High	94.7%	100.0%	96.1%	72/2/2/0	15.1s
#3 gpt-5.6-luna High Codexcodex-cliRun ddc5aa24 Open run Pass rate94.7% 72 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score96.1% 95% interval 84.6–99.1% · 2 run(s) Fixtures37/38 fixtures passed / attempted Outcome mix Pass72 Partial2 Wrong2 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed72 Fixtures solved37/38 Run facts Avg time 15.1s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 84.6–99.1% Suite codex-cli Raw label codex/gpt-5.6-luna@high Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 35/38 passed in linked run perf-find-ents-nearperf-string-table-concatplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
4	Codex	gpt-5.6-terra ddc5aa24	Extra High	94.7%	100.0%	96.1%	72/2/2/0	14.3s
#4 gpt-5.6-terra Extra High Codexcodex-cliRun ddc5aa24 Open run Pass rate94.7% 72 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score96.1% 95% interval 84.6–99.1% · 2 run(s) Fixtures36/38 fixtures passed / attempted Outcome mix Pass72 Partial2 Wrong2 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed72 Fixtures solved36/38 Run facts Avg time 14.3s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 84.6–99.1% Suite codex-cli Raw label codex/gpt-5.6-terra@xhigh Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-looking-atperf-steamid-cacheents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
5	Codex	gpt-5.6-terra ddc5aa24	Low	94.4%	93.4%	96.1%	67/3/1/0	12.8s
#5 gpt-5.6-terra Low Codexcodex-cliRun ddc5aa24 Open run Pass rate94.4% 67 pass · 71 scored Coverage93.4% 71 / 76 evidence-bearing Verified score96.1% 95% interval 84.6–99.1% · 2 run(s) Fixtures36/38 fixtures passed / attempted Outcome mix Pass67 Partial3 Wrong1 Invalid output0 Harness5 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored71 Passed67 Fixtures solved36/38 Run facts Avg time 12.8s Repeat Single-shot Evidence share 71/76 (93.4%) Verified runs 2 95% interval 84.6–99.1% Suite codex-cli Raw label codex/gpt-5.6-terra@low Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 31/38 passed in linked run perf-angle-zeroperf-ents-iteratorperf-for-vs-whileperf-localplayer-cacheperf-looking-atperf-string-table-concatperf-find-ents-nearents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
6	AntiGravity	Gemini 3.5 Flash c99506be	Medium	92.0%	100.0%	94.3%	104/7/2/1	6.1s
#6 Gemini 3.5 Flash Medium AntiGravityagyRun c99506be Open run Pass rate92.0% 104 pass · 113 scored Coverage100.0% 114 / 114 evidence-bearing Verified score94.3% 95% interval 82.1–98.4% · 3 run(s) Fixtures36/38 fixtures passed / attempted Outcome mix Pass104 Partial7 Wrong2 Invalid output1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored113 Passed104 Fixtures solved36/38 Run facts Avg time 6.1s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 82.1–98.4% Suite agy Raw label agy/Gemini 3.5 Flash (Medium) Run id c99506be-e7cc-4900-90b0-2a059597aa89 Latest run fixtures 29/38 passed in linked run perf-string-table-concatperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-seq-tablesperf-angle-zeroperf-find-ents-nearperf-localplayer-cacheperf-looking-atperf-steamid-cacheents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-meta-vs-argperf-msg-vs-printperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
7	Codex	gpt-5.6-luna ddc5aa24	Extra High	93.4%	100.0%	94.1%	71/1/4/0	14.0s
#7 gpt-5.6-luna Extra High Codexcodex-cliRun ddc5aa24 Open run Pass rate93.4% 71 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score94.1% 95% interval 81.8–98.3% · 2 run(s) Fixtures37/38 fixtures passed / attempted Outcome mix Pass71 Partial1 Wrong4 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed71 Fixtures solved37/38 Run facts Avg time 14.0s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 81.8–98.3% Suite codex-cli Raw label codex/gpt-5.6-luna@xhigh Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 35/38 passed in linked run perf-find-ents-nearperf-string-table-concatperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
8	Codex	gpt-5.5 ddc5aa24	Medium	92.1%	100.0%	93.4%	70/2/4/0	11.0s
#8 gpt-5.5 Medium Codexcodex-cliRun ddc5aa24 Open run Pass rate92.1% 70 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score93.4% 95% interval 80.9–97.9% · 2 run(s) Fixtures36/38 fixtures passed / attempted Outcome mix Pass70 Partial2 Wrong4 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed70 Fixtures solved36/38 Run facts Avg time 11.0s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 80.9–97.9% Suite codex-cli Raw label codex/gpt-5.5@medium Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 35/38 passed in linked run perf-find-ents-nearperf-string-table-concatperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
9	Codex	gpt-5.6-luna ddc5aa24	Max	92.1%	100.0%	93.4%	70/2/4/0	18.9s
#9 gpt-5.6-luna Max Codexcodex-cliRun ddc5aa24 Open run Pass rate92.1% 70 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score93.4% 95% interval 80.9–97.9% · 2 run(s) Fixtures38/38 fixtures passed / attempted Outcome mix Pass70 Partial2 Wrong4 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed70 Fixtures solved38/38 Run facts Avg time 18.9s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 80.9–97.9% Suite codex-cli Raw label codex/gpt-5.6-luna@max Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-find-ents-nearperf-string-table-concatperf-angle-zeroperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
10	Codex	gpt-5.6-sol ddc5aa24	Max	92.1%	100.0%	92.8%	70/1/5/0	15.4s
#10 gpt-5.6-sol Max Codexcodex-cliRun ddc5aa24 Open run Pass rate92.1% 70 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score92.8% 95% interval 80.1–97.6% · 2 run(s) Fixtures37/38 fixtures passed / attempted Outcome mix Pass70 Partial1 Wrong5 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed70 Fixtures solved37/38 Run facts Avg time 15.4s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 80.1–97.6% Suite codex-cli Raw label codex/gpt-5.6-sol@max Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 34/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
11	Codex	gpt-5.4-mini ddc5aa24	Extra High	90.3%	99.1%	92.1%	102/4/7/0	29.0s
#11 gpt-5.4-mini Extra High Codexcodex-cliRun ddc5aa24 Open run Pass rate90.3% 102 pass · 113 scored Coverage99.1% 113 / 114 evidence-bearing Verified score92.1% 95% interval 79.2–97.3% · 3 run(s) Fixtures37/38 fixtures passed / attempted Outcome mix Pass102 Partial4 Wrong7 Invalid output0 Harness1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored113 Passed102 Fixtures solved37/38 Run facts Avg time 29.0s Repeat Single-shot Evidence share 113/114 (99.1%) Verified runs 3 95% interval 79.2–97.3% Suite codex-cli Raw label codex/gpt-5.4-mini@xhigh Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 32/38 passed in linked run perf-ply-table-indexperf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
12	Codex	gpt-5.5 ddc5aa24	High	93.3%	100.0%	92.1%	70/0/5/1	15.6s
#12 gpt-5.5 High Codexcodex-cliRun ddc5aa24 Open run Pass rate93.3% 70 pass · 75 scored Coverage100.0% 76 / 76 evidence-bearing Verified score92.1% 95% interval 79.2–97.3% · 2 run(s) Fixtures38/38 fixtures passed / attempted Outcome mix Pass70 Partial0 Wrong5 Invalid output1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored75 Passed70 Fixtures solved38/38 Run facts Avg time 15.6s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 79.2–97.3% Suite codex-cli Raw label codex/gpt-5.5@high Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 30/38 passed in linked run perf-ents-iteratorperf-find-ents-nearperf-hook-onceperf-string-table-concatperf-msg-vs-printperf-table-randomperf-angle-zeroperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-x-times-xplayer-iterator
13	Codex	gpt-5.6-sol ddc5aa24	Extra High	90.8%	100.0%	91.4%	69/1/6/0	15.9s
#13 gpt-5.6-sol Extra High Codexcodex-cliRun ddc5aa24 Open run Pass rate90.8% 69 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score91.4% 95% interval 78.3–96.9% · 2 run(s) Fixtures35/38 fixtures passed / attempted Outcome mix Pass69 Partial1 Wrong6 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed69 Fixtures solved35/38 Run facts Avg time 15.9s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 78.3–96.9% Suite codex-cli Raw label codex/gpt-5.6-sol@xhigh Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-table-hasvalueperf-find-ents-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
14	Codex	gpt-5.5 ddc5aa24	Extra High	89.5%	100.0%	91.4%	68/3/5/0	16.5s
#14 gpt-5.5 Extra High Codexcodex-cliRun ddc5aa24 Open run Pass rate89.5% 68 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score91.4% 95% interval 78.3–96.9% · 2 run(s) Fixtures36/38 fixtures passed / attempted Outcome mix Pass68 Partial3 Wrong5 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed68 Fixtures solved36/38 Run facts Avg time 16.5s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 78.3–96.9% Suite codex-cli Raw label codex/gpt-5.5@xhigh Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-find-ents-nearperf-string-table-concatperf-ply-table-indexperf-angle-zeroperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
15	Codex	gpt-5.6-sol ddc5aa24	High	89.5%	100.0%	91.4%	68/3/5/0	14.1s
#15 gpt-5.6-sol High Codexcodex-cliRun ddc5aa24 Open run Pass rate89.5% 68 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score91.4% 95% interval 78.3–96.9% · 2 run(s) Fixtures35/38 fixtures passed / attempted Outcome mix Pass68 Partial3 Wrong5 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed68 Fixtures solved35/38 Run facts Avg time 14.1s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 78.3–96.9% Suite codex-cli Raw label codex/gpt-5.6-sol@high Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-for-vs-whileperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
16	AntiGravity	Gemini 3.1 Pro c99506be	High	90.3%	90.4%	90.8%	93/4/6/0	12.9s
#16 Gemini 3.1 Pro High AntiGravityagyRun c99506be Open run Pass rate90.3% 93 pass · 103 scored Coverage90.4% 103 / 114 evidence-bearing Verified score90.8% 95% interval 77.5–96.6% · 3 run(s) Fixtures36/38 fixtures passed / attempted Outcome mix Pass93 Partial4 Wrong6 Invalid output0 Harness11 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored103 Passed93 Fixtures solved36/38 Run facts Avg time 12.9s Repeat Single-shot Evidence share 103/114 (90.4%) Verified runs 3 95% interval 77.5–96.6% Suite agy Raw label agy/Gemini 3.1 Pro (High) Run id c99506be-e7cc-4900-90b0-2a059597aa89 Latest run fixtures 22/38 passed in linked run perf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iteratorperf-darkrpvarperf-seq-tablesperf-find-ents-nearperf-localplayer-cacheperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysql
17	Codex	gpt-5.4 ddc5aa24	Extra High	86.8%	100.0%	90.8%	99/9/6/0	19.4s
#17 gpt-5.4 Extra High Codexcodex-cliRun ddc5aa24 Open run Pass rate86.8% 99 pass · 114 scored Coverage100.0% 114 / 114 evidence-bearing Verified score90.8% 95% interval 77.5–96.6% · 3 run(s) Fixtures36/38 fixtures passed / attempted Outcome mix Pass99 Partial9 Wrong6 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored114 Passed99 Fixtures solved36/38 Run facts Avg time 19.4s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 77.5–96.6% Suite codex-cli Raw label codex/gpt-5.4@xhigh Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 31/38 passed in linked run perf-ents-iteratorperf-find-ents-nearperf-hook-onceperf-string-table-concatperf-ply-table-indexperf-angle-zeroperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
18	AntiGravity	Gemini 3.1 Pro c99506be	Low	87.6%	99.1%	90.4%	99/6/8/0	11.3s
#18 Gemini 3.1 Pro Low AntiGravityagyRun c99506be Open run Pass rate87.6% 99 pass · 113 scored Coverage99.1% 113 / 114 evidence-bearing Verified score90.4% 95% interval 77.0–96.3% · 3 run(s) Fixtures35/38 fixtures passed / attempted Outcome mix Pass99 Partial6 Wrong8 Invalid output0 Harness1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored113 Passed99 Fixtures solved35/38 Run facts Avg time 11.3s Repeat Single-shot Evidence share 113/114 (99.1%) Verified runs 3 95% interval 77.0–96.3% Suite agy Raw label agy/Gemini 3.1 Pro (Low) Run id c99506be-e7cc-4900-90b0-2a059597aa89 Latest run fixtures 31/38 passed in linked run ents-iteratorperf-find-ents-nearperf-string-table-concatperf-localplayer-cacheperf-looking-atperf-steamid-cacheplayer-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
19	Codex	gpt-5.6-sol ddc5aa24	Low	86.8%	100.0%	90.1%	66/5/5/0	8.6s
#19 gpt-5.6-sol Low Codexcodex-cliRun ddc5aa24 Open run Pass rate86.8% 66 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score90.1% 95% interval 76.7–96.2% · 2 run(s) Fixtures34/38 fixtures passed / attempted Outcome mix Pass66 Partial5 Wrong5 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed66 Fixtures solved34/38 Run facts Avg time 8.6s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 76.7–96.2% Suite codex-cli Raw label codex/gpt-5.6-sol@low Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-for-vs-whileperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
20	Codex	gpt-5.6-terra ddc5aa24	Medium	86.8%	100.0%	90.1%	66/5/5/0	13.3s
#20 gpt-5.6-terra Medium Codexcodex-cliRun ddc5aa24 Open run Pass rate86.8% 66 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score90.1% 95% interval 76.7–96.2% · 2 run(s) Fixtures35/38 fixtures passed / attempted Outcome mix Pass66 Partial5 Wrong5 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed66 Fixtures solved35/38 Run facts Avg time 13.3s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 76.7–96.2% Suite codex-cli Raw label codex/gpt-5.6-terra@medium Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-looking-atperf-ply-table-indexents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
21	OpenCode	hy3-free d59ee9c0	High	86.8%	100.0%	90.1%	66/5/5/0	11.8s
#21 hy3-free High OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate86.8% 66 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score90.1% 95% interval 76.7–96.2% · 2 run(s) Fixtures34/38 fixtures passed / attempted Outcome mix Pass66 Partial5 Wrong5 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed66 Fixtures solved34/38 Run facts Avg time 11.8s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 76.7–96.2% Suite opencode-zen Raw label opencode/hy3-free@high Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 33/38 passed in linked run perf-find-ents-nearperf-string-table-concatperf-table-hasvalueperf-angle-zeroperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
22	Codex	gpt-5.4-mini ddc5aa24	High	86.8%	100.0%	89.5%	99/6/9/0	12.0s
#22 gpt-5.4-mini High Codexcodex-cliRun ddc5aa24 Open run Pass rate86.8% 99 pass · 114 scored Coverage100.0% 114 / 114 evidence-bearing Verified score89.5% 95% interval 75.9–95.8% · 3 run(s) Fixtures35/38 fixtures passed / attempted Outcome mix Pass99 Partial6 Wrong9 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored114 Passed99 Fixtures solved35/38 Run facts Avg time 12.0s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 75.9–95.8% Suite codex-cli Raw label codex/gpt-5.4-mini@high Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
23	Codex	gpt-5.6-sol ddc5aa24	Medium	86.8%	100.0%	89.5%	66/4/6/0	11.2s
#23 gpt-5.6-sol Medium Codexcodex-cliRun ddc5aa24 Open run Pass rate86.8% 66 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score89.5% 95% interval 75.9–95.8% · 2 run(s) Fixtures33/38 fixtures passed / attempted Outcome mix Pass66 Partial4 Wrong6 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed66 Fixtures solved33/38 Run facts Avg time 11.2s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 75.9–95.8% Suite codex-cli Raw label codex/gpt-5.6-sol@medium Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-for-vs-whileperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
24	OpenRouter	hy3 59029ed6	Low	86.3%	64.9%	89.2%	63/6/4/1	11.2s
#24 hy3 Low OpenRouteropenrouterRun 59029ed6 Open run Pass rate86.3% 63 pass · 73 scored Coverage64.9% 74 / 114 evidence-bearing Verified score89.2% 95% interval 75.3–95.7% · 3 run(s) Fixtures33/37 fixtures passed / attempted Outcome mix Pass63 Partial6 Wrong4 Invalid output1 Harness40 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored73 Passed63 Fixtures solved33/37 Run facts Avg time 11.2s Repeat Single-shot Evidence share 74/114 (64.9%) Verified runs 3 95% interval 75.3–95.7% Suite openrouter Raw label openrouter/tencent/hy3:free@low Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 30/38 passed in linked run perf-string-table-concatperf-surface-drawrectperf-table-hasvalueperf-ply-table-indexplayer-iteratorperf-ents-iteratorperf-find-ents-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
25	AntiGravity	Gemini 3.5 Flash c99506be	Low	83.2%	99.1%	89.0%	94/13/6/0	5.2s
#25 Gemini 3.5 Flash Low AntiGravityagyRun c99506be Open run Pass rate83.2% 94 pass · 113 scored Coverage99.1% 113 / 114 evidence-bearing Verified score89.0% 95% interval 75.3–95.6% · 3 run(s) Fixtures35/38 fixtures passed / attempted Outcome mix Pass94 Partial13 Wrong6 Invalid output0 Harness1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored113 Passed94 Fixtures solved35/38 Run facts Avg time 5.2s Repeat Single-shot Evidence share 113/114 (99.1%) Verified runs 3 95% interval 75.3–95.6% Suite agy Raw label agy/Gemini 3.5 Flash (Low) Run id c99506be-e7cc-4900-90b0-2a059597aa89 Latest run fixtures 29/38 passed in linked run perf-string-table-concatperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-table-hasvalueperf-angle-zeroperf-find-ents-nearperf-localplayer-cacheperf-looking-atperf-steamid-cacheents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-meta-vs-argperf-msg-vs-printperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
26	Codex	gpt-5.6-luna ddc5aa24	Medium	89.3%	100.0%	88.8%	67/1/7/1	9.8s
#26 gpt-5.6-luna Medium Codexcodex-cliRun ddc5aa24 Open run Pass rate89.3% 67 pass · 75 scored Coverage100.0% 76 / 76 evidence-bearing Verified score88.8% 95% interval 75.1–95.5% · 2 run(s) Fixtures37/38 fixtures passed / attempted Outcome mix Pass67 Partial1 Wrong7 Invalid output1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored75 Passed67 Fixtures solved37/38 Run facts Avg time 9.8s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 75.1–95.5% Suite codex-cli Raw label codex/gpt-5.6-luna@medium Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 34/38 passed in linked run perf-find-ents-nearperf-seq-tablesperf-string-table-concatperf-angle-zeroents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
27	Codex	gpt-5.6-terra ddc5aa24	High	85.5%	100.0%	88.8%	65/5/6/0	11.5s
#27 gpt-5.6-terra High Codexcodex-cliRun ddc5aa24 Open run Pass rate85.5% 65 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score88.8% 95% interval 75.1–95.5% · 2 run(s) Fixtures33/38 fixtures passed / attempted Outcome mix Pass65 Partial5 Wrong6 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed65 Fixtures solved33/38 Run facts Avg time 11.5s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 75.1–95.5% Suite codex-cli Raw label codex/gpt-5.6-terra@high Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 32/38 passed in linked run perf-hook-onceperf-string-table-concatperf-angle-zeroperf-darkrpvarperf-find-ents-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
28	OpenRouter	hy3 59029ed6	High	83.8%	64.9%	88.5%	62/7/5/0	17.2s
#28 hy3 High OpenRouteropenrouterRun 59029ed6 Open run Pass rate83.8% 62 pass · 74 scored Coverage64.9% 74 / 114 evidence-bearing Verified score88.5% 95% interval 74.5–95.3% · 3 run(s) Fixtures32/37 fixtures passed / attempted Outcome mix Pass62 Partial7 Wrong5 Invalid output0 Harness40 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored74 Passed62 Fixtures solved32/37 Run facts Avg time 17.2s Repeat Single-shot Evidence share 74/114 (64.9%) Verified runs 3 95% interval 74.5–95.3% Suite openrouter Raw label openrouter/tencent/hy3:free@high Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 29/38 passed in linked run perf-angle-zeroperf-string-table-concatperf-table-hasvalueperf-disttosqrperf-ply-table-indexplayer-iteratorperf-ents-iteratorperf-find-ents-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
29	Codex	gpt-5.4-mini ddc5aa24	Medium	88.4%	100.0%	88.2%	99/3/10/2	13.3s
#29 gpt-5.4-mini Medium Codexcodex-cliRun ddc5aa24 Open run Pass rate88.4% 99 pass · 112 scored Coverage100.0% 114 / 114 evidence-bearing Verified score88.2% 95% interval 74.3–95.0% · 3 run(s) Fixtures36/38 fixtures passed / attempted Outcome mix Pass99 Partial3 Wrong10 Invalid output2 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored112 Passed99 Fixtures solved36/38 Run facts Avg time 13.3s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 74.3–95.0% Suite codex-cli Raw label codex/gpt-5.4-mini@medium Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 31/38 passed in linked run perf-find-ents-nearperf-seq-tablesperf-string-table-concatperf-angle-zeroperf-localplayer-cacheperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
30	Codex	gpt-5.4 ddc5aa24	High	85.8%	100.0%	88.2%	97/7/9/1	13.3s
#30 gpt-5.4 High Codexcodex-cliRun ddc5aa24 Open run Pass rate85.8% 97 pass · 113 scored Coverage100.0% 114 / 114 evidence-bearing Verified score88.2% 95% interval 74.3–95.0% · 3 run(s) Fixtures38/38 fixtures passed / attempted Outcome mix Pass97 Partial7 Wrong9 Invalid output1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored113 Passed97 Fixtures solved38/38 Run facts Avg time 13.3s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 74.3–95.0% Suite codex-cli Raw label codex/gpt-5.4@high Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 31/38 passed in linked run perf-ents-iteratorperf-find-ents-nearperf-hook-onceperf-string-table-concatperf-ply-table-indexperf-angle-zeroperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
31	OpenCode	hy3-free d59ee9c0	Medium	89.2%	100.0%	88.2%	66/2/6/2	12.9s
#31 hy3-free Medium OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate89.2% 66 pass · 74 scored Coverage100.0% 76 / 76 evidence-bearing Verified score88.2% 95% interval 74.3–95.0% · 2 run(s) Fixtures37/38 fixtures passed / attempted Outcome mix Pass66 Partial2 Wrong6 Invalid output2 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored74 Passed66 Fixtures solved37/38 Run facts Avg time 12.9s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 74.3–95.0% Suite opencode-zen Raw label opencode/hy3-free@medium Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 32/38 passed in linked run perf-find-ents-nearperf-ply-table-indexperf-seq-tablesperf-string-table-concatperf-find-players-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
32	Codex	gpt-5.4 ddc5aa24	Low	84.2%	100.0%	87.7%	96/8/10/0	7.7s
#32 gpt-5.4 Low Codexcodex-cliRun ddc5aa24 Open run Pass rate84.2% 96 pass · 114 scored Coverage100.0% 114 / 114 evidence-bearing Verified score87.7% 95% interval 73.7–94.8% · 3 run(s) Fixtures34/38 fixtures passed / attempted Outcome mix Pass96 Partial8 Wrong10 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored114 Passed96 Fixtures solved34/38 Run facts Avg time 7.7s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 73.7–94.8% Suite codex-cli Raw label codex/gpt-5.4@low Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 30/38 passed in linked run perf-ents-iteratorperf-find-ents-nearperf-hook-onceperf-string-table-concatperf-angle-zeroperf-darkrpvarperf-ply-table-indexperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
33	OpenCode	nemotron-3-ultra-free d59ee9c0	High	87.7%	100.0%	86.8%	64/4/5/3	12.6s
#33 nemotron-3-ultra-free High OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate87.7% 64 pass · 73 scored Coverage100.0% 76 / 76 evidence-bearing Verified score86.8% 95% interval 72.7–94.3% · 2 run(s) Fixtures33/38 fixtures passed / attempted Outcome mix Pass64 Partial4 Wrong5 Invalid output3 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored73 Passed64 Fixtures solved33/38 Run facts Avg time 12.6s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 72.7–94.3% Suite opencode-zen Raw label opencode/nemotron-3-ultra-free@high Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 31/38 passed in linked run perf-seq-tablesperf-string-table-concatperf-angle-zeroperf-player-cache-hooksperf-table-hasvalueperf-find-ents-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
34	Codex	gpt-5.5 ddc5aa24	Low	84.2%	100.0%	86.8%	64/4/8/0	10.8s
#34 gpt-5.5 Low Codexcodex-cliRun ddc5aa24 Open run Pass rate84.2% 64 pass · 76 scored Coverage100.0% 76 / 76 evidence-bearing Verified score86.8% 95% interval 72.7–94.3% · 2 run(s) Fixtures34/38 fixtures passed / attempted Outcome mix Pass64 Partial4 Wrong8 Invalid output0 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored76 Passed64 Fixtures solved34/38 Run facts Avg time 10.8s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 72.7–94.3% Suite codex-cli Raw label codex/gpt-5.5@low Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 32/38 passed in linked run perf-find-ents-nearperf-string-table-concatperf-darkrpvarperf-angle-zeroperf-for-vs-whileperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
35	Codex	gpt-5.4 ddc5aa24	Medium	84.1%	100.0%	86.4%	95/7/11/1	11.8s
#35 gpt-5.4 Medium Codexcodex-cliRun ddc5aa24 Open run Pass rate84.1% 95 pass · 113 scored Coverage100.0% 114 / 114 evidence-bearing Verified score86.4% 95% interval 72.2–94.0% · 3 run(s) Fixtures34/38 fixtures passed / attempted Outcome mix Pass95 Partial7 Wrong11 Invalid output1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored113 Passed95 Fixtures solved34/38 Run facts Avg time 11.8s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 72.2–94.0% Suite codex-cli Raw label codex/gpt-5.4@medium Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 33/38 passed in linked run perf-find-ents-nearperf-string-table-concatperf-darkrpvarperf-angle-zeroperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
36	AntiGravity	Claude Sonnet 4.6 c4a59a56	Thinking	85.1%	50.0%	86.2%	63/5/6/2	8.3s
#36 Claude Sonnet 4.6 Thinking AntiGravityagyRun c4a59a56 Open run Pass rate85.1% 63 pass · 74 scored Coverage50.0% 76 / 152 evidence-bearing Verified score86.2% 95% interval 71.9–93.8% · 4 run(s) Fixtures32/38 fixtures passed / attempted Outcome mix Pass63 Partial5 Wrong6 Invalid output2 Harness76 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts152 Scored74 Passed63 Fixtures solved32/38 Run facts Avg time 8.3s Repeat Single-shot Evidence share 76/152 (50.0%) Verified runs 4 95% interval 71.9–93.8% Suite agy Raw label agy/Claude Sonnet 4.6 (Thinking) Run id c4a59a56-eb25-4438-afae-b13af70d3626 Latest run fixtures 29/38 passed in linked run perf-find-ents-nearperf-seq-tablesperf-string-table-concatperf-angle-zeroperf-pairs-ipairs-forperf-ply-table-indexperf-table-hasvalueperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-plain-lua-vs-fnperf-player-cache-hooksperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
37	OpenCode	mimo-v2.5-free d59ee9c0	Low	79.7%	100.0%	86.2%	59/13/2/2	13.5s
#37 mimo-v2.5-free Low OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate79.7% 59 pass · 74 scored Coverage100.0% 76 / 76 evidence-bearing Verified score86.2% 95% interval 71.9–93.8% · 2 run(s) Fixtures32/38 fixtures passed / attempted Outcome mix Pass59 Partial13 Wrong2 Invalid output2 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored74 Passed59 Fixtures solved32/38 Run facts Avg time 13.5s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 71.9–93.8% Suite opencode-zen Raw label opencode/mimo-v2.5-free@low Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 30/38 passed in linked run perf-hook-onceperf-sqlite-vs-mysqlperf-string-table-concatperf-angle-zeroperf-darkrpvarperf-find-ents-nearperf-for-vs-whileperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
38	Codex	gpt-5.6-luna ddc5aa24	Low	85.3%	100.0%	85.5%	64/2/9/1	7.3s
#38 gpt-5.6-luna Low Codexcodex-cliRun ddc5aa24 Open run Pass rate85.3% 64 pass · 75 scored Coverage100.0% 76 / 76 evidence-bearing Verified score85.5% 95% interval 71.1–93.4% · 2 run(s) Fixtures35/38 fixtures passed / attempted Outcome mix Pass64 Partial2 Wrong9 Invalid output1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored75 Passed64 Fixtures solved35/38 Run facts Avg time 7.3s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 71.1–93.4% Suite codex-cli Raw label codex/gpt-5.6-luna@low Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 32/38 passed in linked run perf-seq-tablesperf-string-table-concatperf-angle-zeroperf-darkrpvarperf-find-ents-nearperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
39	OpenCode	hy3-free d59ee9c0	Low	85.1%	100.0%	85.5%	63/4/7/2	12.1s
#39 hy3-free Low OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate85.1% 63 pass · 74 scored Coverage100.0% 76 / 76 evidence-bearing Verified score85.5% 95% interval 71.1–93.4% · 2 run(s) Fixtures35/38 fixtures passed / attempted Outcome mix Pass63 Partial4 Wrong7 Invalid output2 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored74 Passed63 Fixtures solved35/38 Run facts Avg time 12.1s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 71.1–93.4% Suite opencode-zen Raw label opencode/hy3-free@low Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 30/38 passed in linked run perf-find-ents-nearperf-string-table-concatperf-table-hasvalueperf-angle-zeroperf-ply-table-indexperf-sqlite-vs-mysqlperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
40	OpenCode	nemotron-3-ultra-free d59ee9c0	Low	83.6%	100.0%	84.9%	61/7/5/3	13.7s
#40 nemotron-3-ultra-free Low OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate83.6% 61 pass · 73 scored Coverage100.0% 76 / 76 evidence-bearing Verified score84.9% 95% interval 70.3–93.0% · 2 run(s) Fixtures34/38 fixtures passed / attempted Outcome mix Pass61 Partial7 Wrong5 Invalid output3 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored73 Passed61 Fixtures solved34/38 Run facts Avg time 13.7s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 70.3–93.0% Suite opencode-zen Raw label opencode/nemotron-3-ultra-free@low Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 30/38 passed in linked run perf-find-ents-nearperf-for-vs-whileperf-seq-tablesperf-string-table-concatperf-angle-zeroperf-looking-atperf-table-emptyplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
41	OpenRouter	nemotron-3-ultra-550b-a55b 59029ed6	High	83.0%	98.2%	84.0%	88/13/5/6	7.1s
#41 nemotron-3-ultra-550b-a55b High OpenRouteropenrouterRun 59029ed6 Open run Pass rate83.0% 88 pass · 106 scored Coverage98.2% 112 / 114 evidence-bearing Verified score84.0% 95% interval 69.3–92.4% · 3 run(s) Fixtures32/38 fixtures passed / attempted Outcome mix Pass88 Partial13 Wrong5 Invalid output6 Harness2 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored106 Passed88 Fixtures solved32/38 Run facts Avg time 7.1s Repeat Single-shot Evidence share 112/114 (98.2%) Verified runs 3 95% interval 69.3–92.4% Suite openrouter Raw label openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@high Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 27/38 passed in linked run perf-angle-zeroperf-find-ents-nearperf-for-vs-whileperf-plain-lua-vs-fnperf-seq-tablesperf-sqlite-vs-mysqlperf-string-table-concatperf-ents-iteratorperf-looking-atperf-ply-table-indexplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-player-cache-hooksperf-setdrawcolor-splitperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
42	OpenRouter	nemotron-3-ultra-550b-a55b 59029ed6	Medium	83.0%	97.4%	83.8%	88/9/9/5	9.5s
#42 nemotron-3-ultra-550b-a55b Medium OpenRouteropenrouterRun 59029ed6 Open run Pass rate83.0% 88 pass · 106 scored Coverage97.4% 111 / 114 evidence-bearing Verified score83.8% 95% interval 69.1–92.3% · 3 run(s) Fixtures32/38 fixtures passed / attempted Outcome mix Pass88 Partial9 Wrong9 Invalid output5 Harness3 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored106 Passed88 Fixtures solved32/38 Run facts Avg time 9.5s Repeat Single-shot Evidence share 111/114 (97.4%) Verified runs 3 95% interval 69.1–92.3% Suite openrouter Raw label openrouter/nvidia/nemotron-3-ultra-550b-a55b:free@medium Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 30/38 passed in linked run perf-find-ents-nearperf-plain-lua-vs-fnperf-ply-table-indexperf-seq-tablesperf-string-table-concatperf-angle-zeroperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-player-cache-hooksperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
43	OpenRouter	hy3 59029ed6	None	79.7%	66.7%	83.6%	59/9/6/2	3.9s
#43 hy3 None OpenRouteropenrouterRun 59029ed6 Open run Pass rate79.7% 59 pass · 74 scored Coverage66.7% 76 / 114 evidence-bearing Verified score83.6% 95% interval 68.8–92.1% · 3 run(s) Fixtures30/38 fixtures passed / attempted Outcome mix Pass59 Partial9 Wrong6 Invalid output2 Harness38 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored74 Passed59 Fixtures solved30/38 Run facts Avg time 3.9s Repeat Single-shot Evidence share 76/114 (66.7%) Verified runs 3 95% interval 68.8–92.1% Suite openrouter Raw label openrouter/tencent/hy3:free@none Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 31/38 passed in linked run perf-string-table-concatperf-angle-zeroperf-surface-textperf-ents-iteratorperf-find-ents-nearperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
44	Codex	gpt-5.4-mini ddc5aa24	Low	83.5%	100.0%	82.5%	91/6/12/5	8.8s
#44 gpt-5.4-mini Low Codexcodex-cliRun ddc5aa24 Open run Pass rate83.5% 91 pass · 109 scored Coverage100.0% 114 / 114 evidence-bearing Verified score82.5% 95% interval 67.6–91.4% · 3 run(s) Fixtures33/38 fixtures passed / attempted Outcome mix Pass91 Partial6 Wrong12 Invalid output5 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored109 Passed91 Fixtures solved33/38 Run facts Avg time 8.8s Repeat Single-shot Evidence share 114/114 (100.0%) Verified runs 3 95% interval 67.6–91.4% Suite codex-cli Raw label codex/gpt-5.4-mini@low Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 32/38 passed in linked run perf-seq-tablesperf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
45	OpenCode	deepseek-v4-flash-free d59ee9c0	High	79.5%	98.7%	80.3%	58/6/9/2	12.3s
#45 deepseek-v4-flash-free High OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate79.5% 58 pass · 73 scored Coverage98.7% 75 / 76 evidence-bearing Verified score80.3% 95% interval 65.1–89.9% · 2 run(s) Fixtures31/38 fixtures passed / attempted Outcome mix Pass58 Partial6 Wrong9 Invalid output2 Harness1 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored73 Passed58 Fixtures solved31/38 Run facts Avg time 12.3s Repeat Single-shot Evidence share 75/76 (98.7%) Verified runs 2 95% interval 65.1–89.9% Suite opencode-zen Raw label opencode/deepseek-v4-flash-free@high Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 31/38 passed in linked run perf-seq-tablesperf-string-table-concatperf-table-count-hashperf-angle-zeroperf-find-ents-nearperf-looking-atperf-steamid-cacheents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-surface-drawrectperf-surface-textperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
46	OpenCode	nemotron-3-ultra-free d59ee9c0	Medium	77.1%	96.1%	79.6%	54/9/7/3	13.2s
#46 nemotron-3-ultra-free Medium OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate77.1% 54 pass · 70 scored Coverage96.1% 73 / 76 evidence-bearing Verified score79.6% 95% interval 64.4–89.4% · 2 run(s) Fixtures32/38 fixtures passed / attempted Outcome mix Pass54 Partial9 Wrong7 Invalid output3 Harness3 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored70 Passed54 Fixtures solved32/38 Run facts Avg time 13.2s Repeat Single-shot Evidence share 73/76 (96.1%) Verified runs 2 95% interval 64.4–89.4% Suite opencode-zen Raw label opencode/nemotron-3-ultra-free@medium Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 28/38 passed in linked run perf-pairs-ipairs-forperf-seq-tablesperf-string-table-concatperf-ply-table-indexperf-angle-zeroperf-ents-iteratorperf-find-ents-nearperf-looking-atperf-table-emptyplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-plain-lua-vs-fnperf-player-cache-hooksperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
47	OpenRouter	north-mini-code 59029ed6	—	79.0%	72.8%	79.5%	64/6/11/2	11.8s
#47 north-mini-code OpenRouteropenrouterRun 59029ed6 Open run Pass rate79.0% 64 pass · 81 scored Coverage72.8% 83 / 114 evidence-bearing Verified score79.5% 95% interval 63.6–89.6% · 3 run(s) Fixtures29/35 fixtures passed / attempted Outcome mix Pass64 Partial6 Wrong11 Invalid output2 Harness31 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored81 Passed64 Fixtures solved29/35 Run facts Avg time 11.8s Repeat Single-shot Evidence share 83/114 (72.8%) Verified runs 3 95% interval 63.6–89.6% Suite openrouter Raw label openrouter/cohere/north-mini-code:free Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 19/38 passed in linked run perf-angle-zeroperf-find-ents-nearperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-looking-atperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-ply-table-indexperf-seq-tablesperf-sqlite-vs-mysqlperf-string-table-concatperf-surface-textperf-table-emptyperf-x-times-xperf-darkrpvarperf-meta-vs-argplayer-iteratorperf-ents-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-localplayer-cacheperf-msg-vs-printperf-player-cache-hooksperf-setdrawcolor-splitperf-steamid-cacheperf-surface-drawrectperf-table-count-hashperf-table-hasvalueperf-table-literalperf-table-random
48	OpenRouter	nemotron-3-super-120b-a12b 59029ed6	Low	86.2%	66.7%	78.3%	56/8/1/11	9.8s
#48 nemotron-3-super-120b-a12b Low OpenRouteropenrouterRun 59029ed6 Open run Pass rate86.2% 56 pass · 65 scored Coverage66.7% 76 / 114 evidence-bearing Verified score78.3% 95% interval 62.9–88.4% · 3 run(s) Fixtures32/38 fixtures passed / attempted Outcome mix Pass56 Partial8 Wrong1 Invalid output11 Harness38 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored65 Passed56 Fixtures solved32/38 Run facts Avg time 9.8s Repeat Single-shot Evidence share 76/114 (66.7%) Verified runs 3 95% interval 62.9–88.4% Suite openrouter Raw label openrouter/nvidia/nemotron-3-super-120b-a12b:free@low Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 3/38 passed in linked run hook-addisvalidperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iteratorperf-angle-zeroents-iteratornet-receiversperf-config-var
49	Codex	gpt-5.3-codex-spark ddc5aa24	Low	80.3%	100.0%	78.3%	57/5/9/5	10.5s
#49 gpt-5.3-codex-spark Low Codexcodex-cliRun ddc5aa24 Open run Pass rate80.3% 57 pass · 71 scored Coverage100.0% 76 / 76 evidence-bearing Verified score78.3% 95% interval 62.9–88.4% · 2 run(s) Fixtures32/38 fixtures passed / attempted Outcome mix Pass57 Partial5 Wrong9 Invalid output5 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored71 Passed57 Fixtures solved32/38 Run facts Avg time 10.5s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 62.9–88.4% Suite codex-cli Raw label codex/gpt-5.3-codex-spark@low Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 27/38 passed in linked run perf-ents-iteratorperf-find-ents-nearperf-seq-tablesperf-string-table-concatperf-surface-drawrectperf-ply-table-indexperf-angle-zeroperf-for-vs-whileperf-local-colorperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
50	OpenCode	deepseek-v4-flash-free d59ee9c0	Medium	78.1%	100.0%	78.3%	57/5/11/3	11.5s
#50 deepseek-v4-flash-free Medium OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate78.1% 57 pass · 73 scored Coverage100.0% 76 / 76 evidence-bearing Verified score78.3% 95% interval 62.9–88.4% · 2 run(s) Fixtures31/38 fixtures passed / attempted Outcome mix Pass57 Partial5 Wrong11 Invalid output3 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored73 Passed57 Fixtures solved31/38 Run facts Avg time 11.5s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 62.9–88.4% Suite opencode-zen Raw label opencode/deepseek-v4-flash-free@medium Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 31/38 passed in linked run perf-hook-onceperf-seq-tablesperf-string-table-concatperf-angle-zeroperf-find-ents-nearperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
51	OpenCode	deepseek-v4-flash-free d59ee9c0	Low	76.7%	100.0%	78.3%	56/7/10/3	12.6s
#51 deepseek-v4-flash-free Low OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate76.7% 56 pass · 73 scored Coverage100.0% 76 / 76 evidence-bearing Verified score78.3% 95% interval 62.9–88.4% · 2 run(s) Fixtures31/38 fixtures passed / attempted Outcome mix Pass56 Partial7 Wrong10 Invalid output3 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored73 Passed56 Fixtures solved31/38 Run facts Avg time 12.6s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 62.9–88.4% Suite opencode-zen Raw label opencode/deepseek-v4-flash-free@low Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 29/38 passed in linked run perf-for-vs-whileperf-seq-tablesperf-string-table-concatperf-msg-vs-printperf-ply-table-indexperf-angle-zeroperf-find-ents-nearperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
52	Codex	gpt-5.3-codex-spark ddc5aa24	High	80.0%	100.0%	77.0%	56/5/9/6	11.1s
#52 gpt-5.3-codex-spark High Codexcodex-cliRun ddc5aa24 Open run Pass rate80.0% 56 pass · 70 scored Coverage100.0% 76 / 76 evidence-bearing Verified score77.0% 95% interval 61.5–87.5% · 2 run(s) Fixtures30/38 fixtures passed / attempted Outcome mix Pass56 Partial5 Wrong9 Invalid output6 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored70 Passed56 Fixtures solved30/38 Run facts Avg time 11.1s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 61.5–87.5% Suite codex-cli Raw label codex/gpt-5.3-codex-spark@high Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 26/38 passed in linked run perf-ents-iteratorperf-find-ents-nearperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-string-table-concatperf-surface-drawrectperf-table-hasvalueperf-msg-vs-printperf-angle-zeroperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
53	Codex	gpt-5.3-codex-spark ddc5aa24	Medium	78.9%	100.0%	76.3%	56/4/11/5	5.4s
#53 gpt-5.3-codex-spark Medium Codexcodex-cliRun ddc5aa24 Open run Pass rate78.9% 56 pass · 71 scored Coverage100.0% 76 / 76 evidence-bearing Verified score76.3% 95% interval 60.8–87.0% · 2 run(s) Fixtures30/38 fixtures passed / attempted Outcome mix Pass56 Partial4 Wrong11 Invalid output5 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored71 Passed56 Fixtures solved30/38 Run facts Avg time 5.4s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 60.8–87.0% Suite codex-cli Raw label codex/gpt-5.3-codex-spark@medium Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 25/38 passed in linked run perf-angle-zeroperf-ents-iteratorperf-find-ents-nearperf-seq-tablesperf-setdrawcolor-splitperf-string-table-concatperf-surface-drawrectperf-table-hasvalueperf-msg-vs-printperf-player-cache-hooksperf-ply-table-indexperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
54	OpenCode	mimo-v2.5-free d59ee9c0	High	80.6%	100.0%	75.7%	54/7/6/9	14.1s
#54 mimo-v2.5-free High OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate80.6% 54 pass · 67 scored Coverage100.0% 76 / 76 evidence-bearing Verified score75.7% 95% interval 60.1–86.5% · 2 run(s) Fixtures31/38 fixtures passed / attempted Outcome mix Pass54 Partial7 Wrong6 Invalid output9 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored67 Passed54 Fixtures solved31/38 Run facts Avg time 14.1s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 60.1–86.5% Suite opencode-zen Raw label opencode/mimo-v2.5-free@high Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 28/38 passed in linked run perf-config-varperf-ents-iteratorperf-find-ents-nearperf-hook-onceperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-seq-tablesperf-string-table-concatperf-angle-zeroperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-darkrpvarperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-player-cache-hooksperf-ply-table-indexperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
55	OpenCode	mimo-v2.5-free d59ee9c0	Medium	73.2%	100.0%	75.0%	52/10/9/5	13.8s
#55 mimo-v2.5-free Medium OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate73.2% 52 pass · 71 scored Coverage100.0% 76 / 76 evidence-bearing Verified score75.0% 95% interval 59.4–86.0% · 2 run(s) Fixtures29/38 fixtures passed / attempted Outcome mix Pass52 Partial10 Wrong9 Invalid output5 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored71 Passed52 Fixtures solved29/38 Run facts Avg time 13.8s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 59.4–86.0% Suite opencode-zen Raw label opencode/mimo-v2.5-free@medium Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 24/38 passed in linked run perf-for-vs-whileperf-msg-vs-printperf-seq-tablesperf-string-table-concatperf-table-hasvalueplayer-iteratorents-iteratorperf-darkrpvarperf-find-players-nearperf-meta-vs-argperf-ply-table-indexperf-angle-zeroperf-find-ents-nearperf-looking-athook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-ents-iteratorperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
56	Codex	gpt-5.3-codex-spark ddc5aa24	Extra High	76.1%	100.0%	74.3%	54/5/12/5	12.4s
#56 gpt-5.3-codex-spark Extra High Codexcodex-cliRun ddc5aa24 Open run Pass rate76.1% 54 pass · 71 scored Coverage100.0% 76 / 76 evidence-bearing Verified score74.3% 95% interval 58.7–85.5% · 2 run(s) Fixtures30/38 fixtures passed / attempted Outcome mix Pass54 Partial5 Wrong12 Invalid output5 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored71 Passed54 Fixtures solved30/38 Run facts Avg time 12.4s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 58.7–85.5% Suite codex-cli Raw label codex/gpt-5.3-codex-spark@xhigh Run id ddc5aa24-4d6c-4889-9348-24344c06a4ec Latest run fixtures 27/38 passed in linked run perf-angle-zeroperf-ents-iteratorperf-find-ents-nearperf-seq-tablesperf-setdrawcolor-splitperf-string-table-concatperf-table-hasvalueperf-ply-table-indexperf-find-players-nearperf-looking-atplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
57	OpenCode	big-pickle d59ee9c0	—	78.8%	100.0%	73.7%	52/8/6/10	15.1s
#57 big-pickle OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate78.8% 52 pass · 66 scored Coverage100.0% 76 / 76 evidence-bearing Verified score73.7% 95% interval 58.0–85.0% · 2 run(s) Fixtures32/38 fixtures passed / attempted Outcome mix Pass52 Partial8 Wrong6 Invalid output10 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored66 Passed52 Fixtures solved32/38 Run facts Avg time 15.1s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 58.0–85.0% Suite opencode-zen Raw label opencode/big-pickle Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 25/38 passed in linked run hook-addperf-darkrpvarperf-seq-tablesperf-sqlite-vs-mysqlperf-string-table-concatperf-table-hasvalueperf-table-literalperf-player-cache-hooksperf-angle-zeroperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-looking-atents-iteratorisvalidnet-receiversperf-config-varperf-disttosqrperf-ents-iteratorperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-ply-table-indexperf-setdrawcolor-splitperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-randomperf-x-times-xplayer-iterator
58	OpenCode	deepseek-v4-flash-free d59ee9c0	Max	68.5%	100.0%	69.7%	50/6/17/3	12.7s
#58 deepseek-v4-flash-free Max OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate68.5% 50 pass · 73 scored Coverage100.0% 76 / 76 evidence-bearing Verified score69.7% 95% interval 53.9–82.0% · 2 run(s) Fixtures29/38 fixtures passed / attempted Outcome mix Pass50 Partial6 Wrong17 Invalid output3 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored73 Passed50 Fixtures solved29/38 Run facts Avg time 12.7s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 53.9–82.0% Suite opencode-zen Raw label opencode/deepseek-v4-flash-free@max Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 31/38 passed in linked run perf-string-table-concatperf-ply-table-indexperf-angle-zeroperf-find-ents-nearperf-looking-atperf-steamid-cacheplayer-iteratorents-iteratorhook-addisvalidnet-receiversperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
59	OpenRouter	nemotron-3-super-120b-a12b 59029ed6	Medium	79.8%	90.4%	68.9%	67/13/4/19	18.0s
#59 nemotron-3-super-120b-a12b Medium OpenRouteropenrouterRun 59029ed6 Open run Pass rate79.8% 67 pass · 84 scored Coverage90.4% 103 / 114 evidence-bearing Verified score68.9% 95% interval 53.0–81.3% · 3 run(s) Fixtures28/38 fixtures passed / attempted Outcome mix Pass67 Partial13 Wrong4 Invalid output19 Harness11 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored84 Passed67 Fixtures solved28/38 Run facts Avg time 18.0s Repeat Single-shot Evidence share 103/114 (90.4%) Verified runs 3 95% interval 53.0–81.3% Suite openrouter Raw label openrouter/nvidia/nemotron-3-super-120b-a12b:free@medium Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 24/38 passed in linked run perf-local-colorperf-local-vs-globalents-iteratorhook-addisvalidnet-receiversperf-find-ents-nearperf-for-vs-whileperf-msg-vs-printplayer-iteratorperf-angle-zeroperf-ents-iteratorperf-looking-atperf-table-emptyperf-config-varperf-darkrpvarperf-disttosqrperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-localplayer-cacheperf-meta-vs-argperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
60	OpenRouter	laguna-xs-2.1 59029ed6	—	62.9%	65.8%	66.4%	44/11/15/5	5.2s
#60 laguna-xs-2.1 OpenRouteropenrouterRun 59029ed6 Open run Pass rate62.9% 44 pass · 70 scored Coverage65.8% 75 / 114 evidence-bearing Verified score66.4% 95% interval 50.5–79.3% · 3 run(s) Fixtures26/38 fixtures passed / attempted Outcome mix Pass44 Partial11 Wrong15 Invalid output5 Harness39 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored70 Passed44 Fixtures solved26/38 Run facts Avg time 5.2s Repeat Single-shot Evidence share 75/114 (65.8%) Verified runs 3 95% interval 50.5–79.3% Suite openrouter Raw label openrouter/poolside/laguna-xs-2.1:free Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 23/38 passed in linked run perf-looking-atperf-seq-tablesperf-setdrawcolor-splitperf-string-table-concatperf-table-emptyperf-darkrpvarperf-disttosqrperf-meta-vs-argperf-msg-vs-printents-iteratorperf-angle-zeroperf-ents-iteratorperf-find-ents-nearperf-for-vs-whileplayer-iteratorhook-addisvalidnet-receiversperf-config-varperf-find-players-nearperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
61	OpenCode	north-mini-code-free d59ee9c0	High	67.2%	100.0%	59.2%	41/8/12/15	11.6s
#61 north-mini-code-free High OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate67.2% 41 pass · 61 scored Coverage100.0% 76 / 76 evidence-bearing Verified score59.2% 95% interval 43.5–73.3% · 2 run(s) Fixtures25/38 fixtures passed / attempted Outcome mix Pass41 Partial8 Wrong12 Invalid output15 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored61 Passed41 Fixtures solved25/38 Run facts Avg time 11.6s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 43.5–73.3% Suite opencode-zen Raw label opencode/north-mini-code-free@high Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 23/38 passed in linked run perf-angle-zeroperf-ents-iteratorperf-pairs-ipairs-forperf-seq-tablesperf-setdrawcolor-splitperf-string-table-concatperf-table-emptyperf-table-hasvalueplayer-iteratorperf-darkrpvarperf-meta-vs-argperf-ply-table-indexperf-find-ents-nearperf-local-colorperf-looking-atents-iteratorhook-addisvalidnet-receiversperf-config-varperf-disttosqrperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-vs-globalperf-localplayer-cacheperf-msg-vs-printperf-plain-lua-vs-fnperf-player-cache-hooksperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-literalperf-table-randomperf-x-times-x
62	OpenCode	north-mini-code-free d59ee9c0	Medium	58.1%	100.0%	54.6%	36/11/15/14	9.9s
#62 north-mini-code-free Medium OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate58.1% 36 pass · 62 scored Coverage100.0% 76 / 76 evidence-bearing Verified score54.6% 95% interval 39.1–69.3% · 2 run(s) Fixtures23/38 fixtures passed / attempted Outcome mix Pass36 Partial11 Wrong15 Invalid output14 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored62 Passed36 Fixtures solved23/38 Run facts Avg time 9.9s Repeat Single-shot Evidence share 76/76 (100.0%) Verified runs 2 95% interval 39.1–69.3% Suite opencode-zen Raw label opencode/north-mini-code-free@medium Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 20/38 passed in linked run ents-iteratorperf-for-vs-whileperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-seq-tablesperf-setdrawcolor-splitperf-string-table-concatperf-table-emptyperf-darkrpvarperf-disttosqrperf-ply-table-indexperf-surface-textplayer-iteratorperf-angle-zeroperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-looking-athook-addisvalidnet-receiversperf-config-varperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-meta-vs-argperf-msg-vs-printperf-player-cache-hooksperf-sqlite-vs-mysqlperf-steamid-cacheperf-surface-drawrectperf-table-count-hashperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
63	OpenRouter	nemotron-3-nano-30b-a3b 59029ed6	—	66.2%	78.1%	53.5%	49/11/14/15	10.9s
#63 nemotron-3-nano-30b-a3b OpenRouteropenrouterRun 59029ed6 Open run Pass rate66.2% 49 pass · 74 scored Coverage78.1% 89 / 114 evidence-bearing Verified score53.5% 95% interval 38.1–68.3% · 3 run(s) Fixtures22/38 fixtures passed / attempted Outcome mix Pass49 Partial11 Wrong14 Invalid output15 Harness25 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts114 Scored74 Passed49 Fixtures solved22/38 Run facts Avg time 10.9s Repeat Single-shot Evidence share 89/114 (78.1%) Verified runs 3 95% interval 38.1–68.3% Suite openrouter Raw label openrouter/nvidia/nemotron-3-nano-30b-a3b:free Run id 59029ed6-30eb-4d96-ad76-21deda2a06fa Latest run fixtures 16/38 passed in linked run isvalidperf-angle-zeroperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-steamid-cacheperf-string-table-concatperf-surface-textperf-table-emptyperf-darkrpvarperf-meta-vs-argperf-sqlite-vs-mysqlperf-find-players-nearperf-hook-onceperf-looking-atplayer-iteratorents-iteratorhook-addnet-receiversperf-config-varperf-for-vs-whileperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-setdrawcolor-splitperf-surface-drawrectperf-table-count-hashperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-x
64	OpenCode	north-mini-code-free d59ee9c0	Low	58.5%	96.1%	48.0%	31/8/14/20	12.4s
#64 north-mini-code-free Low OpenCodeopencode-zenRun d59ee9c0 Open run Pass rate58.5% 31 pass · 53 scored Coverage96.1% 73 / 76 evidence-bearing Verified score48.0% 95% interval 33.1–63.3% · 2 run(s) Fixtures21/38 fixtures passed / attempted Outcome mix Pass31 Partial8 Wrong14 Invalid output20 Harness3 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored53 Passed31 Fixtures solved21/38 Run facts Avg time 12.4s Repeat Single-shot Evidence share 73/76 (96.1%) Verified runs 2 95% interval 33.1–63.3% Suite opencode-zen Raw label opencode/north-mini-code-free@low Run id d59ee9c0-dbbb-4508-aad0-897bdc9849c2 Latest run fixtures 17/38 passed in linked run hook-addperf-darkrpvarperf-surface-textents-iteratorperf-config-varperf-local-vs-globalperf-looking-atperf-pairs-ipairs-forperf-seq-tablesperf-setdrawcolor-splitperf-string-table-concatperf-table-count-hashperf-table-hasvalueperf-disttosqrperf-meta-vs-argperf-msg-vs-printperf-ply-table-indexperf-sqlite-vs-mysqlplayer-iteratorperf-angle-zeroperf-find-ents-nearisvalidnet-receiversperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-localplayer-cacheperf-plain-lua-vs-fnperf-player-cache-hooksperf-steamid-cacheperf-surface-drawrectperf-table-emptyperf-table-literalperf-table-randomperf-x-times-x
65	OpenRouter	nemotron-nano-9b-v2 38c62341	—	50.0%	92.1%	3.4%	2/1/1/66	10.5s
#65 nemotron-nano-9b-v2 OpenRouteropenrouterRun 38c62341 Open run Pass rate50.0% 2 pass · 4 scored Coverage92.1% 70 / 76 evidence-bearing Verified score3.4% 95% interval 0.7–14.8% · 2 run(s) Fixtures2/37 fixtures passed / attempted Outcome mix Pass2 Partial1 Wrong1 Invalid output66 Harness6 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored4 Passed2 Fixtures solved2/37 Run facts Avg time 10.5s Repeat Single-shot Evidence share 70/76 (92.1%) Verified runs 2 95% interval 0.7–14.8% Suite openrouter Raw label openrouter/nvidia/nemotron-nano-9b-v2:free Run id 38c62341-b58b-4e7d-9756-3fc0c03c77c9 Latest run fixtures 2/38 passed in linked run ents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomplayer-iteratorperf-find-players-nearperf-pairs-ipairs-forperf-x-times-x
66	OpenRouter	laguna-m.1 38c62341	—	0.0%	46.1%	0.0%	0/0/0/35	9.0s
#66 laguna-m.1 OpenRouteropenrouterRun 38c62341 Open run Pass rate0.0% 0 pass · 0 scored Coverage46.1% 35 / 76 evidence-bearing Verified score0.0% 95% interval 0.0–9.9% · 2 run(s) Fixtures0/35 fixtures passed / attempted Outcome mix Pass0 Partial0 Wrong0 Invalid output35 Harness41 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored0 Passed0 Fixtures solved0/35 Run facts Avg time 9.0s Repeat Single-shot Evidence share 35/76 (46.1%) Verified runs 2 95% interval 0.0–9.9% Suite openrouter Raw label openrouter/poolside/laguna-m.1:free Run id 38c62341-b58b-4e7d-9756-3fc0c03c77c9 Latest run fixtures 0/38 passed in linked run ents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
—	OpenRouter	gpt-oss-20b 38c62341	High	100.0%	2.6%	100.0%	2/0/0/0	5.8s
Unranked gpt-oss-20b High OpenRouteropenrouterRun 38c62341 Open run Pass rate100.0% 2 pass · 2 scored Coverage2.6% 2 / 76 evidence-bearing Verified score100.0% 95% interval 34.2–100.0% · 2 run(s) Fixtures2/2 fixtures passed / attempted Outcome mix Pass2 Partial0 Wrong0 Invalid output0 Harness74 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored2 Passed2 Fixtures solved2/2 Run facts Avg time 5.8s Repeat Single-shot Evidence share 2/76 (2.6%) Verified runs 2 95% interval 34.2–100.0% Suite openrouter Raw label openrouter/openai/gpt-oss-20b:free@high Run id 38c62341-b58b-4e7d-9756-3fc0c03c77c9 Latest run fixtures 0/38 passed in linked run ents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
—	OpenRouter	gpt-oss-120b 38c62341	High	100.0%	1.3%	100.0%	1/0/0/0	1.3s
Unranked gpt-oss-120b High OpenRouteropenrouterRun 38c62341 Open run Pass rate100.0% 1 pass · 1 scored Coverage1.3% 1 / 76 evidence-bearing Verified score100.0% 95% interval 20.6–100.0% · 2 run(s) Fixtures1/1 fixtures passed / attempted Outcome mix Pass1 Partial0 Wrong0 Invalid output0 Harness75 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored1 Passed1 Fixtures solved1/1 Run facts Avg time 1.3s Repeat Single-shot Evidence share 1/76 (1.3%) Verified runs 2 95% interval 20.6–100.0% Suite openrouter Raw label openrouter/openai/gpt-oss-120b:free@high Run id 38c62341-b58b-4e7d-9756-3fc0c03c77c9 Latest run fixtures 0/38 passed in linked run ents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
—	OpenRouter	gpt-oss-120b 38c62341	Low	100.0%	1.3%	100.0%	1/0/0/0	882ms
Unranked gpt-oss-120b Low OpenRouteropenrouterRun 38c62341 Open run Pass rate100.0% 1 pass · 1 scored Coverage1.3% 1 / 76 evidence-bearing Verified score100.0% 95% interval 20.6–100.0% · 2 run(s) Fixtures1/1 fixtures passed / attempted Outcome mix Pass1 Partial0 Wrong0 Invalid output0 Harness75 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored1 Passed1 Fixtures solved1/1 Run facts Avg time 882ms Repeat Single-shot Evidence share 1/76 (1.3%) Verified runs 2 95% interval 20.6–100.0% Suite openrouter Raw label openrouter/openai/gpt-oss-120b:free@low Run id 38c62341-b58b-4e7d-9756-3fc0c03c77c9 Latest run fixtures 0/38 passed in linked run ents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
—	OpenRouter	gpt-oss-120b 38c62341	Medium	100.0%	1.3%	100.0%	1/0/0/0	1.6s
Unranked gpt-oss-120b Medium OpenRouteropenrouterRun 38c62341 Open run Pass rate100.0% 1 pass · 1 scored Coverage1.3% 1 / 76 evidence-bearing Verified score100.0% 95% interval 20.6–100.0% · 2 run(s) Fixtures1/1 fixtures passed / attempted Outcome mix Pass1 Partial0 Wrong0 Invalid output0 Harness75 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored1 Passed1 Fixtures solved1/1 Run facts Avg time 1.6s Repeat Single-shot Evidence share 1/76 (1.3%) Verified runs 2 95% interval 20.6–100.0% Suite openrouter Raw label openrouter/openai/gpt-oss-120b:free@medium Run id 38c62341-b58b-4e7d-9756-3fc0c03c77c9 Latest run fixtures 0/38 passed in linked run ents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator
—	AntiGravity	Claude Opus 4.6 c4a59a56	Thinking	71.0%	20.4%	83.3%	22/7/2/0	7.1s
Unranked Claude Opus 4.6 Thinking AntiGravityagyRun c4a59a56 Open run Pass rate71.0% 22 pass · 31 scored Coverage20.4% 31 / 152 evidence-bearing Verified score83.3% 95% interval 60.8–94.2% · 4 run(s) Fixtures13/18 fixtures passed / attempted Outcome mix Pass22 Partial7 Wrong2 Invalid output0 Harness121 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts152 Scored31 Passed22 Fixtures solved13/18 Run facts Avg time 7.1s Repeat Single-shot Evidence share 31/152 (20.4%) Verified runs 4 95% interval 60.8–94.2% Suite agy Raw label agy/Claude Opus 4.6 (Thinking) Run id c4a59a56-eb25-4438-afae-b13af70d3626 Latest run fixtures 10/38 passed in linked run perf-find-ents-nearperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iteratorperf-config-varperf-angle-zeroents-iteratorhook-addisvalidnet-receiversperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-players-nearperf-for-vs-whileperf-hook-once
—	OpenRouter	gpt-oss-20b 38c62341	Medium	0.0%	1.3%	0.0%	0/0/1/0	137ms
Unranked gpt-oss-20b Medium OpenRouteropenrouterRun 38c62341 Open run Pass rate0.0% 0 pass · 1 scored Coverage1.3% 1 / 76 evidence-bearing Verified score0.0% 95% interval 0.0–79.3% · 2 run(s) Fixtures0/1 fixtures passed / attempted Outcome mix Pass0 Partial0 Wrong1 Invalid output0 Harness75 Invalid output means the model returned content that broke the answer contract, so it scores zero. Harness failures returned no answer and are excluded from score. Attempt funnel Attempts76 Scored1 Passed0 Fixtures solved0/1 Run facts Avg time 137ms Repeat Single-shot Evidence share 1/76 (1.3%) Verified runs 2 95% interval 0.0–79.3% Suite openrouter Raw label openrouter/openai/gpt-oss-20b:free@medium Run id 38c62341-b58b-4e7d-9756-3fc0c03c77c9 Latest run fixtures 0/38 passed in linked run ents-iteratorhook-addisvalidnet-receiversperf-angle-zeroperf-config-varperf-darkrpvarperf-disttosqrperf-ents-iteratorperf-find-ents-nearperf-find-players-nearperf-for-vs-whileperf-hook-onceperf-hudpaint-cacheperf-local-colorperf-local-vs-globalperf-localplayer-cacheperf-looking-atperf-meta-vs-argperf-msg-vs-printperf-pairs-ipairs-forperf-plain-lua-vs-fnperf-player-cache-hooksperf-ply-table-indexperf-seq-tablesperf-setdrawcolor-splitperf-sqlite-vs-mysqlperf-steamid-cacheperf-string-table-concatperf-surface-drawrectperf-surface-textperf-table-count-hashperf-table-emptyperf-table-hasvalueperf-table-literalperf-table-randomperf-x-times-xplayer-iterator

Verified score aggregates compatible runs. Outcomes = pass / partial / wrong / invalid output; harness failures are excluded. Click a row for evidence and the linked run's fixtures.

How to read this boarddisclaimer · columns · source runs

Ranks aggregate all verified runs in the broadest compatible fixture cohort. Each fixture has equal weight; harness failures are excluded, malformed model answers score zero, and numeric rank requires at least 50% fixture coverage.

Columns

Adapter: Runner used: Codex, AntiGravity, OpenCode, OpenRouter, …
Think: Reasoning effort: Low, Medium, High, Extra High, Max, Ultra. Filter via Think.
Pass %: pass ÷ scored. Invalid output is tracked separately from semantically graded answers.
Cover %: Evidence-bearing attempts divided by scheduled attempts; harness failures are excluded.
Verified score: Cumulative equal-weight fixture score. Harness failures are excluded; malformed returned answers score zero.
Outcomes: pass / partial / wrong / invalid output. Invalid output means the model answered but broke the required response shape.
Time: Average wall time per attempt.

Source runs

0f05b466AntiGravity · 8 models
38c62341OpenRouter · 26 models
3a137d39OpenRouter · 26 models
4524afb8AntiGravity · 8 models
49efc61aAntiGravity · 2 models
55e3cd85Codex · 31 models
59029ed6OpenRouter · 10 models
5aadc831OpenCode · 1 models
6d5384a1Codex · 8 models
70b164dcAntiGravity · 2 models
779e9572OpenRouter · 26 models
7b1432b7OpenRouter · 31 models
824cda9aOpenCode · 17 models
943f32a1OpenRouter · 26 models
994f70d3OpenRouter · 4 models
9e2fc537OpenRouter · 31 models
a9024895OpenRouter · 31 models
c0a6995eOpenRouter · 23 models
c4a59a56AntiGravity · 2 models
c56989a5OpenCode · 17 models
c99506beAntiGravity · 8 models
d59ee9c0OpenCode · 17 models
d5aeb478OpenRouter · 6 models
ddc5aa24Codex · 31 models

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures

Outcome mix

Attempt funnel

Run facts

Latest run fixtures