Cycle 1: c_puct 3->5, dirichlet_eps 0.25->0.4 produced first natural-end wins

50-iter sweep on default.yaml with stronger MCTS exploration:
- c_puct: 3.0 -> 5.0 (UCB weight, more exploration of low-prior actions)
- root_dirichlet_epsilon: 0.25 -> 0.4 (more noise injected at root prior)

Standalone eval at iter 50 (30 games/opponent, all natural-end, timeouts=0):
- vs heuristic-balanced:    W=0/30 S=-70.5  (PA 0.15)
- vs heuristic-aggressive:  W=2/30 S=-65.1  (PA 0.14)  [+10, +4]
- vs heuristic-cautious:    W=1/30 S=-48.0  (PA 0.14)  [+2]

3 natural-end wins vs prev trapfix baseline iter 44 (which had 0 natural
wins + 1 timeout-tie). Stall trap fixed remains true (timeouts=0 in c1).

Trade-off observed: more exploration -> higher variance. Score avg vs
cautious worsened (-32 -> -48), but win events appeared. For the
non-terminal-win objective, exploration win > score-avg loss.

Next: commit to long run (300 iter) with these params before tuning more.
This commit is contained in:
2026-05-11 05:53:54 +09:00
parent f289997c1c
commit 169d4dcb14
2 changed files with 32 additions and 2 deletions
+2 -2
View File
@@ -22,14 +22,14 @@ network:
activation: relu
mcts:
n_simulations: 50
c_puct: 3.0
c_puct: 5.0
max_depth: 200
parallel_simulations: 64
virtual_loss_value: 5.0
eval_n_simulations: 16
rollout_policy: heuristic_balanced
root_dirichlet_alpha: 0.3
root_dirichlet_epsilon: 0.25
root_dirichlet_epsilon: 0.4
temperature:
training: 1.0
eval: 0.0
+30
View File
@@ -0,0 +1,30 @@
#!/bin/bash
# Autonomous cycle eval helper.
# Usage: ./autonomous_cycle_eval.sh <run-prefix> [extra eval args...]
# Finds latest run matching prefix, runs eval --ckpt latest.pt with 30 games,
# and reports: timeouts, natural-end wins per opponent.
set -euo pipefail
PREFIX="${1:-}"
shift || true
if [ -z "$PREFIX" ]; then
echo "usage: $0 <run-prefix> [extra eval args...]"
exit 1
fi
RUN=$(ls -td runs/*${PREFIX}* 2>/dev/null | head -1)
if [ -z "$RUN" ]; then
echo "no run matching ${PREFIX}" >&2
exit 1
fi
CKPT="$RUN/latest.pt"
if [ ! -f "$CKPT" ]; then
echo "no checkpoint at $CKPT" >&2
exit 1
fi
echo "=== eval $CKPT ==="
uv run lost-cities-ismcts eval --ckpt "$CKPT" --games 30 --verbose "$@" 2>&1