Cycle 4: revert use_rollout_value=true, scale network 512x3 -> 768x4
c3 showed use_rollout_value=false alone is not the fix: without the heuristic rollout safety net the random-init network value gives bad MCTS Q early on, the agent plays more (PA 0.10 -> 0.22+) but eats more -20 expedition penalties (score worsened from -57 to -97 avg). Mathematical intuition: in Lost Cities, opening an expedition is a 20-point commitment. Break-even requires rank-sum × (handshakes+1) >= 20. The model has to learn: - which colors to open (based on hand handshake/high-rank holdings) - when to commit vs discard - card-ordering constraints (ascending only) This is a moderately rich value function. 512x3 (~800k params, ~290 input dim) might be undersized. Test capacity hypothesis with 768x4 (~2M params) while keeping the rollout safety net so MCTS Q stays competent. Other params from c1 kept: c_puct=5, virtual_loss=5, dirichlet α=0.3/ε=0.4, parallel_simulations=64, n_simulations=50.
This commit is contained in:
@@ -17,8 +17,8 @@ encoding:
|
||||
slot_aware_playability: true
|
||||
network:
|
||||
kind: mlp
|
||||
hidden_size: 512
|
||||
num_layers: 3
|
||||
hidden_size: 768
|
||||
num_layers: 4
|
||||
activation: relu
|
||||
mcts:
|
||||
n_simulations: 50
|
||||
@@ -28,7 +28,7 @@ mcts:
|
||||
virtual_loss_value: 5.0
|
||||
eval_n_simulations: 16
|
||||
rollout_policy: heuristic_balanced
|
||||
use_rollout_value: false
|
||||
use_rollout_value: true
|
||||
root_dirichlet_alpha: 0.3
|
||||
root_dirichlet_epsilon: 0.4
|
||||
temperature:
|
||||
|
||||
Reference in New Issue
Block a user