Agents & AI

The ranking

Head-to-head games, seats alternated, on seeds never used to tune or train the agents. Ties (2 to 5 % of games) are shared. deni-0.8 is the reference opponent: it is the best heuristic, and like the networks it looks at what a move leaves the opponent.

* Overstated: the search simulates its opponent as v3-multi, so it knew exactly how that opponent would answer (see below).

The game currently plays v3-multi, weakened more or less depending on the difficulty level. The first three agents are measured but not wired in yet: the search takes half a second per move and will have to run off the main thread so the interface does not freeze.

The neural network: five corrections

A value network: it scores the sheet obtained after each possible move, and plays the best-scored one. 735 sparse inputs (checked squares by colour, stars, the opponents' squares) and 61 computed inputs, written in TypeScript in one single place; Python only trains on those bytes, so the game and the training cannot drift apart. A test checks that PyTorch and TypeScript compute the same thing. It lost at first — each step fixes what the previous one measured.

1. 1
   Predict the final score−7.67 points

   Trained on the outcome of simulated games, it predicted the score well and played badly: it had never seen two moves from the same position, and learnt the price of an exclamation point by comparing different games.

2. 2
   Compare the moves of one position−2.42 points

   Each candidate is played out 8 times to the end with the same dice, by v3-multi, and the network learns the gaps between moves. Better, but still beaten — and the diagnosis below shows why.

3. 3
   The network plays the rest itself+0.7 point

   Same simulations, but the network keeps its seat until the end: the data says what a move is worth to it. The first network ahead of v3-multi at a 4-player table — and yet 45% head to head.

4. 4
   Play for the margin, not for your points45% → 58%

   The network has two outputs: its score, and its margin over the best opponent. Same weights, a different output played. Aiming for points makes a slow player who gets cut off; aiming for the margin makes it race: it ends 62% of games instead of 31%.

5. 5
   Games of 2 to 4 players63.7%

   All the data came from 4-player tables, and the number of opponents is a network input: head to head, it was extrapolating. New data at 2, 3 and 4 players: it beats v3-multi at every table size (43.0% per seat at 3, 33.5% at 4).

The diagnosis of step 2, and the most useful lesson of the project. Its moves were as good as v3's if v3 played the rest — exactly what its training data measured, since every simulation handed control back to v3. Played by itself, they were worth less. It was valuing positions for a player better than itself. Two move-by-move checks under v3 had come out null before this one showed it.

And the caveat of step 5: deni-0.8, a heuristic that also looks at the opponent, beats v3-multi almost as much (63.6%). Most of the gain came from the objective — playing for the margin — more than from the network itself.

Dice denial

As the active player, a choice has two effects: what it brings you, and what it takes from the others, who only get the 4 remaining dice. That requires enumerating moves per dice pair: the normal generator merges pairs that are equivalent for you — in more than 8 turns out of 10 — and throws away exactly the information that matters to the others.

The optimum is interior in both cases: denying too much hurts too. For the network, what the remaining dice give the opponent is valued by the same network from their seat, so in the same unit — and with 2 players, a weight of 1 is exactly the logic of the margin: what they gain, I lose. The gain shrinks with more players (46.1% per seat at 3 players, 34.5% at 4, against 43.0 and 33.5 without denial).

Useless once search is added: over the same 400 games, the gap is −0.98 point [−2.29, +0.33]. The search already simulates what the opponent does with the dice left; adding the denial price counts it twice.

Thinking before playing: search

The game allows up to 3 seconds per move; the network takes 1.6 ms. The search keeps the network's 4 best moves, then plays out each one 16 times, 4 turns ahead — the network in its seat, v3-multi for the opponent, the rest of the current turn included — and lets the network estimate the final margin of the position reached. The 4 moves see the same dice, otherwise chance would drown the difference.

The 84% is overstated. In its simulations, the search assumes the opponent plays like v3-multi — which was precisely the opponent of that duel: it knew exactly how it would answer. The other rows are against opponents it does not model, and those are the ones to remember.

At 3 and 4 players, parity would be 33% and 25% per seat: the search reaches 53% and 41%, so the network no longer extrapolates with table size.

Measured time: 0.55 s per move on average head to head, 1.0 s at worst; 0.7 s and 1.4 s at 4 players. A wider search (6 moves, 32 simulations, 6 turns) reached 4.9 s: too slow.

Method

• Headless engine — the rules live in engine/, as plain data, with no Vue or Pinia. The interface, the agents and the training share the same implementation: no agent is tuned on rules other than the ones we play.
• Paired comparisons — at equal game index, every agent gets the same seed, so the same dice on the same grid. The noise is shared and cancels out in the gaps, each reported with its 95% interval.
• Reserved seeds — evaluation duels use seeds that neither tuning nor training ever saw, and results are confirmed on a second, disjoint block.
• Data from the policy that plays — a move is scored by simulating the rest with the agent that will really play it; scoring for another player cost 2.4 points per game.
• Always report the table size — the same network won at 4 players and lost head to head: the number of opponents is a network input.
• Negative results kept — depth-2 expectimax brought nothing for 140× the cost, a 21-parameter heuristic did worse than a 6-parameter one, and denial adds nothing to search. These are measurements, not omissions.