Cycle 4: revert use_rollout_value=true, scale network 512x3 -> 768x4

c3 showed use_rollout_value=false alone is not the fix: without the
heuristic rollout safety net the random-init network value gives bad
MCTS Q early on, the agent plays more (PA 0.10 -> 0.22+) but eats more
-20 expedition penalties (score worsened from -57 to -97 avg).

Mathematical intuition: in Lost Cities, opening an expedition is a
20-point commitment. Break-even requires rank-sum × (handshakes+1) >= 20.
The model has to learn:
  - which colors to open (based on hand handshake/high-rank holdings)
  - when to commit vs discard
  - card-ordering constraints (ascending only)

This is a moderately rich value function. 512x3 (~800k params, ~290
input dim) might be undersized. Test capacity hypothesis with 768x4
(~2M params) while keeping the rollout safety net so MCTS Q stays
competent.

Other params from c1 kept: c_puct=5, virtual_loss=5, dirichlet
α=0.3/ε=0.4, parallel_simulations=64, n_simulations=50.
This commit is contained in:
2026-05-11 07:03:45 +09:00
parent 200129d16d
commit a501a93223
+3 -3
View File
@@ -17,8 +17,8 @@ encoding:
slot_aware_playability: true
network:
kind: mlp
hidden_size: 512
num_layers: 3
hidden_size: 768
num_layers: 4
activation: relu
mcts:
n_simulations: 50
@@ -28,7 +28,7 @@ mcts:
virtual_loss_value: 5.0
eval_n_simulations: 16
rollout_policy: heuristic_balanced
use_rollout_value: false
use_rollout_value: true
root_dirichlet_alpha: 0.3
root_dirichlet_epsilon: 0.4
temperature: