Add multi-process self-play, eval workers, MCTS Cython port
Key changes for ISMCTS speed and correctness: - Cython port: HeuristicBot helpers (`heuristic_cy.pyx` + new `.pxd`) and ISMCTS searcher (`mcts.pyx`) now run as cdef. Both share a fast unified-action path through GameState's C interface to avoid Python round-trips on hot rollout/tree-walk paths. - Multi-process self-play and eval: `workers.py`, `eval_worker.py`, `interleaved_self_play.py`, plus trainer wiring with ProcessPoolExecutor + spawn context. Eval inside `evaluate.py` is parallel per opponent. - ISMCTS-specific eval (`evaluate.py`) runs MCTS at decision time so the metric matches deploy mode; `evaluation.eval_with_mcts` flag preserves backwards-compatible policy-only eval when needed. - Trainer logs progress per phase (self-play start/done, eval per opponent), and value loss is now scaled by `value_scale` so policy and value losses sit on comparable magnitudes. - Compact info-set key (`info_set.py`) using packed-struct format and child-key reuse during MCTS descent to cut per-step canonicalization. Tests: 19 ISMCTS suite passing, including parity (Cython-vs-Python sequential, batched-vs-sequential visit counts, push/pop round-trip). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -24,6 +24,8 @@ mcts:
|
||||
n_simulations: 50
|
||||
c_puct: 1.5
|
||||
max_depth: 200
|
||||
parallel_simulations: 8
|
||||
virtual_loss_value: 1.0
|
||||
temperature:
|
||||
training: 1.0
|
||||
eval: 0.0
|
||||
@@ -32,6 +34,8 @@ training:
|
||||
gradient_steps_per_iter: 10
|
||||
batch_size: 128
|
||||
replay_capacity: 100000
|
||||
interleave_games: 8
|
||||
interleave_max_batch: 64
|
||||
optimization:
|
||||
learning_rate: 0.0003
|
||||
grad_clip: 5.0
|
||||
|
||||
@@ -24,6 +24,8 @@ mcts:
|
||||
n_simulations: 50
|
||||
c_puct: 1.5
|
||||
max_depth: 100
|
||||
parallel_simulations: 8
|
||||
virtual_loss_value: 1.0
|
||||
temperature:
|
||||
training: 1.0
|
||||
eval: 0.0
|
||||
@@ -32,6 +34,8 @@ training:
|
||||
gradient_steps_per_iter: 10
|
||||
batch_size: 128
|
||||
replay_capacity: 50000
|
||||
interleave_games: 8
|
||||
interleave_max_batch: 64
|
||||
optimization:
|
||||
learning_rate: 0.001
|
||||
grad_clip: 5.0
|
||||
|
||||
Reference in New Issue
Block a user