Trap diagnosis: agent learned to stall (avoid opening expeditions, draw
from discard pile to extend deck) until max_steps timeout, then squeak by
on opponents' negative scores. All eval wins were from timeouts; agent
never won a naturally-terminating game. Self-play reinforced this because
timeout games still got a positive value target.
Fixes (no algorithm change, all MCTS hyperparameters or signal shaping):
- Dirichlet noise at root prior (AlphaZero standard, was missing):
mcts.pyx `_expand_with_prior` takes `is_root` flag; root expansion
mixes prior with Dirichlet(α). Callers in interleaved_self_play and
the internal evaluate_and_backup pass `not item.path`.
- Default config strengthens exploration on the 50-sim batched search:
c_puct 1.5 -> 3.0, virtual_loss_value 1.0 -> 5.0, plus new
root_dirichlet_alpha=0.3 / root_dirichlet_epsilon=0.25.
- Self-play timeout signal zeroed: `_finalize_context` sets v_target=0
if context.state is not terminal. Stops the network from learning
"stall = positive value".
New standalone evaluator:
- `lost-cities-ismcts eval` subcommand (eval_checkpoint.py): loads a
checkpoint, runs N games per opponent across a parallel pool, reports
win/score with 95% CIs plus per-game logging via --verbose. Defaults
cover heuristic-balanced/aggressive/cautious (rollout policy isn't in
the training-eval opponent list, so this is the natural way to compare
the trained policy against its rollout target).
Tests (19) still pass; .so rebuilt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
coolguyandClaude Opus 4.7
f289997c1c