Add Dirichlet root noise + standalone eval CLI, fix self-play stall trap
Trap diagnosis: agent learned to stall (avoid opening expeditions, draw from discard pile to extend deck) until max_steps timeout, then squeak by on opponents' negative scores. All eval wins were from timeouts; agent never won a naturally-terminating game. Self-play reinforced this because timeout games still got a positive value target. Fixes (no algorithm change, all MCTS hyperparameters or signal shaping): - Dirichlet noise at root prior (AlphaZero standard, was missing): mcts.pyx `_expand_with_prior` takes `is_root` flag; root expansion mixes prior with Dirichlet(α). Callers in interleaved_self_play and the internal evaluate_and_backup pass `not item.path`. - Default config strengthens exploration on the 50-sim batched search: c_puct 1.5 -> 3.0, virtual_loss_value 1.0 -> 5.0, plus new root_dirichlet_alpha=0.3 / root_dirichlet_epsilon=0.25. - Self-play timeout signal zeroed: `_finalize_context` sets v_target=0 if context.state is not terminal. Stops the network from learning "stall = positive value". New standalone evaluator: - `lost-cities-ismcts eval` subcommand (eval_checkpoint.py): loads a checkpoint, runs N games per opponent across a parallel pool, reports win/score with 95% CIs plus per-game logging via --verbose. Defaults cover heuristic-balanced/aggressive/cautious (rollout policy isn't in the training-eval opponent list, so this is the natural way to compare the trained policy against its rollout target). Tests (19) still pass; .so rebuilt. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -30,6 +30,8 @@ class MctsConfig(StrictModel):
|
||||
virtual_loss_value: float = 1.0
|
||||
eval_with_mcts: bool = True
|
||||
eval_n_simulations: int = 0
|
||||
root_dirichlet_alpha: float = 0.0
|
||||
root_dirichlet_epsilon: float = 0.0
|
||||
|
||||
@field_validator("n_simulations", "max_depth", "parallel_simulations")
|
||||
@classmethod
|
||||
|
||||
Reference in New Issue
Block a user