When the arena lies, do not add more games

The most alarming chart in the early OverGrid arena was not an AI leaderboard. It was a control group.

We put the exact same deterministic Balanced policy in every seat, across 360 games, 30 official maps, 12 fresh seeds, a 400-construction budget, and a full-round 120-turn horizon. It still produced enormous seat differences.

Same brain, wildly different results

Among territory-complete games, the strongest physical seat won far more often than its fair share at every table size:

Table	Expected equal share	Strongest seat	Weakest seat
2 players	50.0%	67.4%	32.6%
3 players	33.3%	47.8%	6.5%
4 players	25.0%	52.2%	7.6%
5 players	20.0%	47.6%	1.6%
6 players	16.7%	40.6%	1.0%

At six players, seat 0 expanded fastest yet won just 1.0% of completed games. Seat 5 expanded slowest, collected the most height, and won 40.6%. Immediate neutral gain had been masquerading as control; late response and conversion were doing the real work.

That suggests a gameplay hypothesis: tempo is not the same as durable position. It is not, on its own, an AI ranking.

The audit found two confounds

The large fairness effect prompted a deeper audit. It found two defects that invalidated the pre-fairness research corpus for current strength claims.

First, bounded move enumeration depended on absolute board coordinates. Opposite homes did not receive rotation-equivalent candidate prefixes. A policy could be perfectly deterministic and still be offered a different tactical menu merely because it sat on the other side of a mirrored board.

Second, flip-war detection omitted cleared black cells and could adjudicate before every player had received the same number of turns. The engine was occasionally judging a race before all runners had crossed the same checkpoint. Not ideal, as race officiating goes.

The correct response was to archive, not massage. We preserved manifests and checksums so reports would not silently rot, but prohibited their use for profile ranking, doctrine promotion, home effects, or compute scaling conclusions.

The replacement test was metamorphic

Annotated reasoningAn unfair control invalidates the leaderboard before another game is scheduled.

Read left to right. Each card names the claim it is allowed to support.

1. ControlClone the same policy

   Place one deterministic policy in every physical seat.

   Personality removed→
2. AlarmSeats still diverge

   Large home and action-order splits remain under an identical brain.

   Stop the ranking→
3. RepairTransform the position

   Rotate homes, relabel players, replay commands, and preserve owed turns.

   Exact metamorphic checks→
4. RestartBuild a fresh baseline

   Archive the contaminated corpus and measure again under the repaired contract.

   Permission to compare

The repaired harness had to pass properties that should be exact, not merely statistically plausible:

• swap player labels: no canonicalized result may change;
• rotate the board and swap homes by 180 degrees: candidate prefix, move, score, winner, and ending must agree;
• change worker count, execution order, or batch order: no replay fingerprint may change;
• replay every command through the reducer: state, ledger, score, and terminal label must recompute exactly;
• finish a round: no player may be denied an owed turn.

The v7 integrity gate passed 32/32 label and frame transforms exactly. Across 1,496 turns and 13,464 replayed reducer commands, there were zero move, score, work, winner, or ending differences. This did not prove that the game is balanced. It proved that a fresh baseline could begin without the two known measurement leaks.

A small warning survived the repair

The new gate also found something much more mundane and much more useful: budget 100 could fail to see a reachable neutral-progress construction late on an Expert board. Two games ran past 500 turns with 78.4% and 87.5% completion. The board was not deadlocked; budget 800 exposed 12–33 progress outcomes and ranked the best one first under the unchanged evaluator.

So the research contract became: budget-100 full games are censored; caps are neither wins nor losses; and a low-budget policy may be acceptable for mechanics tests without being acceptable for resolved-game strength estimates.

Discard is a feature

The difficult part of empirical game AI is not generating a bar chart. It is having the nerve to invalidate one. The archived dataset represented real computation and real engineering time. Keeping it visible, but refusing to use it as evidence, was the research win.

We now treat a large unexplained home, seat, or starter effect as a stop sign. First test the representation, schedule, ending rule, and candidate supply. Only then ask whether an AI is clever.

The gameplay question exposed by the control now has its own reader path: Fast expansion is not control.

<details> <summary>Methods and limits</summary>

The historical seat-control panel contained 360 games: identical Balanced policy at every seat, 12 fresh seeds on all 30 official maps, budget 400, 120-turn full-round horizon. It recorded 248 territory completions, 106 right-censored horizons, and 6 flip-war adjudications, all kept separate. The later integrity suite is a small falsification panel, not a game-balance proof or a current strength estimate.

</details>