The web client now plays borealis (data/models.json), trained on the three-round
match and taking the 501-dim match view rather than a bare round. Only the actor
trunk is exported -- the critic exists to grade moves in training and never plays,
so the graph physically cannot leak the opponent's hand or the deck, which beats
promising not to call it.
The TypeScript match layer and observation mirror match.py and match_obs.py. They
have to agree to the bit: a mismatch throws nowhere, the ONNX policy just consumes
a wrong vector and plays worse for reasons nobody can see. So the port is not
trusted -- generate_match_parity_fixture.py emits 282 positions from real JAX play
(mid-round, both seats, past a roll-over, with a live carry) and the TS output is
checked against them to float32 round-off.
Match mode is a menu toggle. A seed fixes all three deals and the coin flips, so a
match stays a pure function of it. One-deal mode is unchanged from the player's
side; borealis simply sees it as round one at a carry of zero, a position it has
seen a great many times.
Two bugs found by driving the built app in a browser, both silent:
- The result card totalled the round, not the match. It read "-11 : 3" while the
match stood at -96 : 66 -- it would have named the wrong winner. It now headlines
the match total and breaks the round out beneath it.
- Game records were being rejected. The client's schema went to v2 (it now records
which model played; the old records stored the on-screen label, which stops
identifying anything once there are two models) while serve_web_with_logs.py
still only accepted v1, so every record would have 400'd into a console warning.
v1 stays accepted -- the 111 existing games are altair.
npm test and tsc were green through both. Hence web/.claude/skills/verify, which
records the recipe and the selectors so the next session drives the app instead of
re-deriving how.
.gitignore excluded the new model, which would have shipped a 404: the deploy
builds straight from the repo.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
The 39.3M ablation said the privileged critic hurt (switching it off won 0.5160
[0.505, 0.527]). Head to head at 131M, both sides trained identically, it says the
opposite: off *loses*, 0.4633 [0.453, 0.474]. Against league the critic-on model
scores 0.6094 and the critic-off one 0.5526.
The critic earns its keep once there is enough data to fit it -- at 39.3M the
privileged value trunk is underfit and only adds advantage noise. Defaulted back
on, with the small-scale number kept in the docstring as a warning: an ablation at
a budget you do not intend to ship can invert.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
The user asked why we were not simply maximising the three-round total, and they
were right. Each ply now pays the points by which it moved the running match
difference; at gamma=1 that telescopes to the final total, so the objective is
exactly the rulebook's, handed out every ply instead of once 150 plies later.
Three measurements killed the tanh design:
- Rounds are independent (corr(m1,m2)=0.004, corr(m1+m2,m3)=0.05), so a reward
linear in the total decomposes the match into three independent rounds and
carry enters the objective nowhere. The only coupling, the start-player rule,
is worth +0.73 +/- 0.84 points -- indistinguishable from zero.
- Risk attitude, the one thing tanh buys, is worthless here. A policy made to
gamble when it trails by 20 entering round three *loses* to a greedy clone over
6144 duplicate matches (0.482); gambling only at -40 breaks even (0.498). A
marginal wager buys about +1.7 sigma for -2 to -3 expected points. Ceiling on
the whole carry-conditioning idea: under one win-rate point.
- Head to head over 10,000 duplicate matches at equal compute, the linear reward
*beats* tanh(total/12): 0.5859 (CI 0.576-0.596), +20.3 points. Dropping it is
not merely free, it is better -- not because of risk, but because tanh hands a
~150-ply match one saturated +/-1 and leaves all credit assignment to the critic.
The flat carry probe was not exploration collapse: sampled play still opens 5.00
expeditions, entropy settles at 1.36 nats (3.9 effective actions), and the critic
reads carry cleanly (round-three values run -0.87 to +0.86, monotone). The signal
was there; there was nothing to buy with it.
Criterion 1 (a monotone carry response) comes off the gate accordingly -- the
optimal response barely exists in this game. carry stays in the observation: it
costs nothing and the start-player rule keys off it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
Phases 2-4 land together because they all rewrite the same rollout, and doing
them in sequence would mean writing it three times.
Observation adds the four things a match policy cannot play without: carry as a
scalar *and* a binned one-hot (round three is a threshold problem -- "win by 41
or lose" plays nothing like "win by 39" -- and the old score_diff divided by
MAX_ABS_SCORE=780, squashing a decisive 50-point lead to 0.06); the round index;
whose turn it is, which the single-round observation never carried even though
the critic is trained on opponent-turn states; and the deck clock, since a round
ends on the last deck draw and players bend that parity by drawing from discard
piles. Live points per colour are split by hand / discard pile / unseen, because
a discard pile is public and recoverable.
The critic is asymmetric: it gets the opponent's hand and the deck in order, on
a separate trunk so none of it can reach the logits. A test pins that down --
perturbing the privileged input leaves the policy logits bit-identical while
moving the value. Deal luck is what makes a match-terminal reward hard to learn
from, and a state-value baseline may condition on anything action-independent.
Both seats now train. Self-play ran one network on both sides and stop_gradiented
the opponent, throwing away half of every game; each ply now emits a transition
per seat, folded into the batch so each seat keeps an independent GAE chain.
Reward is the match: rounds one and two only bank into carry, and round three
pays tanh(total / terminal_scale). Potential shaping on the running total covers
the early sparsity and anneals out.
match_eval adds the two measurements the plan turns on: duplicate match play
(same deals and coins from both seats -- self-play scores exactly 0.500 with zero
mean lead, so the mirroring cancels deal luck exactly) and the carry probe. On an
untrained net the probe is flat: 4.97 expeditions opened at a 60-point deficit
and at a 60-point lead alike. Breaking that flat line is success criterion 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
Wraps the single-round engine rather than changing it: engine.py is the rules
oracle the TypeScript client is differential-tested against, and the round
itself is unchanged by the match. Only two rules live up here, both from the
Kosmos rulebook -- three rounds decided on the summed total, and "the player who
has more points begins" the next one. That is not alternating, and it is not
what reset_from_order hardcoded, so it takes a first_player argument now.
The rulebook says nothing about an exact tie, so the starter falls back to a
coin flip. A deterministic tie-break would give one seat a standing edge in
symmetric self-play and the agent would learn to steer for it. Round one needs
no special case: carry is (0, 0) there, so the tie branch already yields the
coin, which is exactly the rulebook's arbitrary "oldest player begins".
All randomness -- three deals and three coins -- is drawn in match_reset and
stored in the state, so match_step stays deterministic and needs no PRNG key
threaded through every rollout, eval, and gate body. It also makes a mirrored
match (same deals, seats swapped, same coins) a pure seat relabel, which the
antithetic pairing later depends on.
Tests cover the deck clock (44 deck draws a round, discard draws extend it),
carry banking each round exactly once, the start-player rule across all three
branches, a fair round-one coin, mirror symmetry, and that the running total
does not jump across a round boundary -- the last one matters because potential
shaping will be built on it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
rollout_steps was pinned to MAX_STEPS (400) while a round actually runs
~50 plies, and finished envs were only reset between updates. 83% of every
rollout was spent stepping already-done envs to produce masked-out zeros.
Measured at rollout_steps=400: active steps go 17.3% -> 100%, i.e. 5.8x the
learner actions per update for the same compute.
Resetting in-scan exposes three things that were previously benign:
- compute_gae bootstrapped truncated episodes from zero. That was safe only
because every episode used to terminate inside the scan; now episodes cross
the boundary, so thread V(s_T) through.
- rollout_metrics read final_env and summed rewards along the scan axis, both
of which assume one episode per slot. With several episodes per slot that
silently produces garbage, so aggregate at done boundaries instead.
- league assignments were redrawn only between updates, which would pin a slot
to one seat/opponent across every episode in a scan. Redraw them on reset.
Also anneal potential shaping against learner actions rather than padded scan
steps: the old accounting counted the dead steps, so a 5M-step anneal expired
within two updates of 250. Any earlier evidence that shaping does not help was
gathered with it effectively off.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
coolguyandClaude Opus 4.8