Add multi-process self-play, eval workers, MCTS Cython port

Key changes for ISMCTS speed and correctness:
- Cython port: HeuristicBot helpers (`heuristic_cy.pyx` + new `.pxd`) and
  ISMCTS searcher (`mcts.pyx`) now run as cdef. Both share a fast
  unified-action path through GameState's C interface to avoid Python
  round-trips on hot rollout/tree-walk paths.
- Multi-process self-play and eval: `workers.py`, `eval_worker.py`,
  `interleaved_self_play.py`, plus trainer wiring with ProcessPoolExecutor
  + spawn context. Eval inside `evaluate.py` is parallel per opponent.
- ISMCTS-specific eval (`evaluate.py`) runs MCTS at decision time so the
  metric matches deploy mode; `evaluation.eval_with_mcts` flag preserves
  backwards-compatible policy-only eval when needed.
- Trainer logs progress per phase (self-play start/done, eval per
  opponent), and value loss is now scaled by `value_scale` so policy and
  value losses sit on comparable magnitudes.
- Compact info-set key (`info_set.py`) using packed-struct format and
  child-key reuse during MCTS descent to cut per-step canonicalization.

Tests: 19 ISMCTS suite passing, including parity (Cython-vs-Python
sequential, batched-vs-sequential visit counts, push/pop round-trip).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-11 02:39:39 +09:00
co-authored by Claude Opus 4.7
parent 0999d34277
commit 651175e5bd
17 changed files with 2896 additions and 114 deletions
+4
View File
@@ -24,6 +24,8 @@ mcts:
n_simulations: 50
c_puct: 1.5
max_depth: 200
parallel_simulations: 8
virtual_loss_value: 1.0
temperature:
training: 1.0
eval: 0.0
@@ -32,6 +34,8 @@ training:
gradient_steps_per_iter: 10
batch_size: 128
replay_capacity: 100000
interleave_games: 8
interleave_max_batch: 64
optimization:
learning_rate: 0.0003
grad_clip: 5.0
+4
View File
@@ -24,6 +24,8 @@ mcts:
n_simulations: 50
c_puct: 1.5
max_depth: 100
parallel_simulations: 8
virtual_loss_value: 1.0
temperature:
training: 1.0
eval: 0.0
@@ -32,6 +34,8 @@ training:
gradient_steps_per_iter: 10
batch_size: 128
replay_capacity: 50000
interleave_games: 8
interleave_max_batch: 64
optimization:
learning_rate: 0.001
grad_clip: 5.0