coolguyandClaude Opus 4.7
651175e5bd
Key changes for ISMCTS speed and correctness: - Cython port: HeuristicBot helpers (`heuristic_cy.pyx` + new `.pxd`) and ISMCTS searcher (`mcts.pyx`) now run as cdef. Both share a fast unified-action path through GameState's C interface to avoid Python round-trips on hot rollout/tree-walk paths. - Multi-process self-play and eval: `workers.py`, `eval_worker.py`, `interleaved_self_play.py`, plus trainer wiring with ProcessPoolExecutor + spawn context. Eval inside `evaluate.py` is parallel per opponent. - ISMCTS-specific eval (`evaluate.py`) runs MCTS at decision time so the metric matches deploy mode; `evaluation.eval_with_mcts` flag preserves backwards-compatible policy-only eval when needed. - Trainer logs progress per phase (self-play start/done, eval per opponent), and value loss is now scaled by `value_scale` so policy and value losses sit on comparable magnitudes. - Compact info-set key (`info_set.py`) using packed-struct format and child-key reuse during MCTS descent to cut per-step canonicalization. Tests: 19 ISMCTS suite passing, including parity (Cython-vs-Python sequential, batched-vs-sequential visit counts, push/pop round-trip). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
51 lines
907 B
YAML
51 lines
907 B
YAML
run:
|
|
experiment_name: ismcts-mini
|
|
max_iterations: 50
|
|
seed: 1
|
|
device: cpu
|
|
rules:
|
|
n_colors: 3
|
|
n_ranks: 5
|
|
min_rank: 2
|
|
n_handshakes: 1
|
|
hand_size: 4
|
|
expedition_penalty: -20
|
|
bonus_threshold: 4
|
|
bonus_amount: 20
|
|
encoding:
|
|
derived_playability: true
|
|
slot_aware_playability: true
|
|
network:
|
|
kind: mlp
|
|
hidden_size: 128
|
|
num_layers: 2
|
|
activation: relu
|
|
mcts:
|
|
n_simulations: 50
|
|
c_puct: 1.5
|
|
max_depth: 100
|
|
parallel_simulations: 8
|
|
virtual_loss_value: 1.0
|
|
temperature:
|
|
training: 1.0
|
|
eval: 0.0
|
|
training:
|
|
games_per_iter: 10
|
|
gradient_steps_per_iter: 10
|
|
batch_size: 128
|
|
replay_capacity: 50000
|
|
interleave_games: 8
|
|
interleave_max_batch: 64
|
|
optimization:
|
|
learning_rate: 0.001
|
|
grad_clip: 5.0
|
|
checkpoint:
|
|
save_every: 0
|
|
save_latest: true
|
|
evaluation:
|
|
eval_every: 5
|
|
games: 20
|
|
opponents: [random, discard-only, heuristic-cautious]
|
|
max_steps: 300
|
|
num_workers: 1
|