coolguy
169d4dcb14
50-iter sweep on default.yaml with stronger MCTS exploration: - c_puct: 3.0 -> 5.0 (UCB weight, more exploration of low-prior actions) - root_dirichlet_epsilon: 0.25 -> 0.4 (more noise injected at root prior) Standalone eval at iter 50 (30 games/opponent, all natural-end, timeouts=0): - vs heuristic-balanced: W=0/30 S=-70.5 (PA 0.15) - vs heuristic-aggressive: W=2/30 S=-65.1 (PA 0.14) [+10, +4] - vs heuristic-cautious: W=1/30 S=-48.0 (PA 0.14) [+2] 3 natural-end wins vs prev trapfix baseline iter 44 (which had 0 natural wins + 1 timeout-tie). Stall trap fixed remains true (timeouts=0 in c1). Trade-off observed: more exploration -> higher variance. Score avg vs cautious worsened (-32 -> -48), but win events appeared. For the non-terminal-win objective, exploration win > score-avg loss. Next: commit to long run (300 iter) with these params before tuning more.
57 lines
1.0 KiB
YAML
57 lines
1.0 KiB
YAML
run:
|
|
experiment_name: ismcts-default
|
|
max_iterations: 500
|
|
seed: 1
|
|
device: cuda
|
|
rules:
|
|
n_colors: 5
|
|
n_ranks: 9
|
|
min_rank: 2
|
|
n_handshakes: 3
|
|
hand_size: 8
|
|
expedition_penalty: -20
|
|
bonus_threshold: 8
|
|
bonus_amount: 20
|
|
encoding:
|
|
derived_playability: true
|
|
slot_aware_playability: true
|
|
network:
|
|
kind: mlp
|
|
hidden_size: 512
|
|
num_layers: 3
|
|
activation: relu
|
|
mcts:
|
|
n_simulations: 50
|
|
c_puct: 5.0
|
|
max_depth: 200
|
|
parallel_simulations: 64
|
|
virtual_loss_value: 5.0
|
|
eval_n_simulations: 16
|
|
rollout_policy: heuristic_balanced
|
|
root_dirichlet_alpha: 0.3
|
|
root_dirichlet_epsilon: 0.4
|
|
temperature:
|
|
training: 1.0
|
|
eval: 0.0
|
|
training:
|
|
games_per_iter: 10
|
|
gradient_steps_per_iter: 10
|
|
batch_size: 128
|
|
replay_capacity: 100000
|
|
interleave_games: 8
|
|
interleave_max_batch: 64
|
|
num_workers: 8
|
|
worker_device: cuda
|
|
optimization:
|
|
learning_rate: 0.0003
|
|
grad_clip: 5.0
|
|
checkpoint:
|
|
save_every: 20
|
|
save_latest: true
|
|
evaluation:
|
|
eval_every: 5
|
|
games: 5
|
|
opponents: [random, discard-only, heuristic-cautious]
|
|
max_steps: 500
|
|
num_workers: 8
|