Restore the privileged critic: its ablation flips with scale
The 39.3M ablation said the privileged critic hurt (switching it off won 0.5160 [0.505, 0.527]). Head to head at 131M, both sides trained identically, it says the opposite: off *loses*, 0.4633 [0.453, 0.474]. Against league the critic-on model scores 0.6094 and the critic-off one 0.5526. The critic earns its keep once there is enough data to fit it -- at 39.3M the privileged value trunk is underfit and only adds advantage noise. Defaulted back on, with the small-scale number kept in the docstring as a warning: an ablation at a budget you do not intend to ship can invert. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
This commit is contained in:
@@ -13,6 +13,7 @@ from lost_cities_jax.match_obs import (
|
||||
match_observation,
|
||||
)
|
||||
from lost_cities_jax.match_ppo import (
|
||||
Ablation,
|
||||
create_match_train_state,
|
||||
make_match_rollout_fn,
|
||||
make_match_train_iteration,
|
||||
@@ -54,9 +55,14 @@ def test_the_critic_sees_the_opponents_hand_and_the_actor_does_not():
|
||||
|
||||
|
||||
def test_privileged_input_cannot_move_the_policy_logits():
|
||||
"""Separate trunks, or the critic's view of the deck leaks into play."""
|
||||
"""Separate trunks, or the critic's view of the deck leaks into play.
|
||||
|
||||
The privileged critic is off by default -- it measured negative, see the
|
||||
ablation in the plan -- but the isolation property still has to hold for
|
||||
anyone who switches it on, and for its later reuse as a search evaluator.
|
||||
"""
|
||||
cfg = _cfg()
|
||||
state = create_match_train_state(cfg, jax.random.PRNGKey(1))
|
||||
state = create_match_train_state(cfg, jax.random.PRNGKey(1), Ablation(privileged_critic=True))
|
||||
match = match_reset(jax.random.PRNGKey(2))
|
||||
|
||||
obs = match_observation(match, jnp.int32(0))[None, :]
|
||||
|
||||
Reference in New Issue
Block a user