From 0d35341bbe75ae3977a837693f97e72a01edfb82 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=A0=95=EC=8B=9C=EC=9B=90?= Date: Mon, 11 May 2026 07:56:35 +0900 Subject: [PATCH] Cycle 5 setup: long run with use_rollout_value=false + 768x4 network C4 (768x4 + rollout=True, 150 iter) result: 0/300 natural wins, score avg -85 to -97 vs three heuristic opponents. Bigger network alone did not produce wins; PA shifted up to 0.23-0.25 (similar to c3 without rollout) but agent still loses every natural-end game. Capacity hypothesis rejected: 2.5x more params (~2M vs ~800k) did not break the loss pattern. Score average actually slightly worse than 512x3 baseline. So the bottleneck is not network capacity. Going to the long-run experiment: AlphaZero-correct setup with the network value loop closed. use_rollout_value=false means leaf Q comes from network value head. Training signal: network value learns from actual game outcomes; MCTS uses those values to pick actions; better actions produce better outcomes; cycle closes. 100 iter previously gave essentially the same result as rollout=true (comparing c3 to trapfix baseline). Both are too early in the AlphaZero training curve. Standard AlphaZero papers train 1000s of iterations. Going long: 1000 iter with the current config. Self-play ~9s/iter without rollout, total wall ~150 min for the train phase. Plotting strategy: at iter 200, 500, 1000, run 100-game standalone eval and generate analyze.py plots to visualize trajectory. Network kept at 768x4 since bigger capacity does not actively hurt. --- configs/ismcts/default.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configs/ismcts/default.yaml b/configs/ismcts/default.yaml index d9b8e28..bd1a3ed 100644 --- a/configs/ismcts/default.yaml +++ b/configs/ismcts/default.yaml @@ -28,7 +28,7 @@ mcts: virtual_loss_value: 5.0 eval_n_simulations: 16 rollout_policy: heuristic_balanced - use_rollout_value: true + use_rollout_value: false root_dirichlet_alpha: 0.3 root_dirichlet_epsilon: 0.4 temperature: