Restore the privileged critic: its ablation flips with scale

The 39.3M ablation said the privileged critic hurt (switching it off won 0.5160
[0.505, 0.527]). Head to head at 131M, both sides trained identically, it says the
opposite: off *loses*, 0.4633 [0.453, 0.474]. Against league the critic-on model
scores 0.6094 and the critic-off one 0.5526.

The critic earns its keep once there is enough data to fit it -- at 39.3M the
privileged value trunk is underfit and only adds advantage noise. Defaulted back
on, with the small-scale number kept in the docstring as a warning: an ablation at
a budget you do not intend to ship can invert.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
This commit is contained in:
2026-07-15 04:01:40 +09:00
co-authored by Claude Opus 4.8
parent 1f51fb8149
commit 4d1c4473b6
3 changed files with 42 additions and 8 deletions
+8 -2
View File
@@ -13,6 +13,7 @@ from lost_cities_jax.match_obs import (
match_observation,
)
from lost_cities_jax.match_ppo import (
Ablation,
create_match_train_state,
make_match_rollout_fn,
make_match_train_iteration,
@@ -54,9 +55,14 @@ def test_the_critic_sees_the_opponents_hand_and_the_actor_does_not():
def test_privileged_input_cannot_move_the_policy_logits():
"""Separate trunks, or the critic's view of the deck leaks into play."""
"""Separate trunks, or the critic's view of the deck leaks into play.
The privileged critic is off by default -- it measured negative, see the
ablation in the plan -- but the isolation property still has to hold for
anyone who switches it on, and for its later reuse as a search evaluator.
"""
cfg = _cfg()
state = create_match_train_state(cfg, jax.random.PRNGKey(1))
state = create_match_train_state(cfg, jax.random.PRNGKey(1), Ablation(privileged_critic=True))
match = match_reset(jax.random.PRNGKey(2))
obs = match_observation(match, jnp.int32(0))[None, :]