# Deep CFR Selectivity Investigation

Last updated: 2026-05-10

## Current conclusion

The current Deep CFR baseline is not mainly blocked by model size or LCFR
time-weighting. The strongest signal so far is a selectivity failure: the model
does not reliably learn to distinguish good first opens from bad first opens.

The `outcome_sampling_epsilon=0.05` setting produced a short-run improvement
around 200 iterations, but the improvement did not hold through 500 iterations.
Weighting ablations did not recover the 200-iteration peak, so further LCFR
alpha tuning is lower priority than directly inspecting or changing the
first-open advantage target.

## Baseline symptoms

The 512x3 dense-eval baseline showed improving training losses, but the main
game-quality metrics against `heuristic_cautious` did not improve enough to
indicate a useful policy.

Observed pattern:

- Advantage loss can move in the expected direction while eval quality remains
  poor.
- Bad first-open behavior stays high.
- Open quality, measured by score per opened color, stays negative.
- Current-policy evaluation did not rescue the result, so the issue is not only
  average-policy lag.

## Closed hypotheses

### More outcome sampling helps, but only up to a point

Short 200-iteration ablations with `outcome_unsampled_regret=zero`:

| setting | safe strict score diff | win rate | bad open rate | score/opened color |
| --- | ---: | ---: | ---: | ---: |
| `epsilon=0.20` | -57.87 | 0.07 | 0.901 | -8.40 |
| `epsilon=0.10` | -58.66 | 0.05 | 0.913 | -8.63 |
| `epsilon=0.05` | -40.01 | 0.12 | 0.893 | -6.25 |
| `epsilon=0.02` | -53.55 | - | - | - |

Conclusion: `epsilon=0.05` was the best short-run candidate. Lowering to
`0.02` was worse, and larger values were also worse.

### regret_matching_epsilon = 1e-4 was the zero-pit countermeasure (prior repo)

이전 레포(`../coolrl`)에서 zero-pit 발생 당시 시도한 epsilon 튜닝 기록
(commit `5a98855`, `da0d4cd`, 2026-05-06). 이건 `outcome_sampling_epsilon`이
아니라 `regret_matching_epsilon`이라는 별도 파라미터.

- **eps1e3** (`regret_matching_epsilon=1.0e-3`): zero-pit timeout은 줄였지만
  safe 계열 avg_diff 개선 X. selectivity floor를 과하게 만들 가능성 의심.
- **eps1e4** (`regret_matching_epsilon=1.0e-4`): floor 더 낮춰서 219 iter
  수동 중단. random 승률 0.86 / score_diff +32, safe 승률 0.03~0.05 /
  score_diff -57~-74. zero-pit timeout은 거의 안 나오지만 selectivity
  실패는 그대로.

`1.0e-4`가 그때 best로 채택되어 현재 `default.yaml`의 기본값으로 남아 있음
(`regret_matching_epsilon: 0.0001`). R0, R1 양쪽 모두 이 설정 사용 중.

**이 lever는 이미 당겨져 있다.** R1에서 zero-pit이 다시 나타난 것은
floor가 풀려서가 아니라, opponent=discard_only가 "do nothing" 균형을
*수학적으로* 안전하게 만든 별개 원인. epsilon 재조정으로 풀리는 구조가 아님.

### past-self 풀만으로는 trap 못 깸 (prior repo)

이전 레포(`../coolrl`) `full_depth` 실험 (commit `33e0368`, 2026-05-06).

설정 (anchor 없음, 순수 past-self pool):

```yaml
self_play_league:
  current_weight: 0.5
  recent_weight: 0.3
  older_weight: 0.2
  max_snapshots: 20
  recent_window: 5
```

가설: opponent로 본인 최신만 쓰는 self-mirror 평형이 selectivity emergence를
막는다면, 과거 자아 snapshot을 섞으면 평형이 깨진다.

결과 (iter 322 종료):

- safe 상대 opened_colors: **4.94~4.96** (거의 전색 opening)
- 5-color opening 빈도: 91% → 93% (iter 110→180, 감소 없음)
- safe avg_diff: eps1e4 baseline 대비 +25점 회복 (recovery skill은 emerge)
- terminal rate 100%, node cutoff 0% (truncation 정상)

판정 (commit 메시지 그대로): "recovery는 self-play로 emerge했지만 selectivity는
현재 league 평형 안에서 emerge하지 않는다."

함의 — past-self를 섞어도 본인 과거 자아는 같은 trap policy의 시간축 평행이동일
뿐이라 self-mirror 평형이 사실상 그대로 유지된다. selectivity는 self-play 가족
내부 다양성으로 풀리지 않는다.

### self_play_league에 heuristic anchor 0.15 주입은 trap을 깨지 못함 (prior repo)

이전 레포(`../coolrl`) `anchor_safe015` 실험 (commit `279d726`, `a77464b`,
2026-05-06).

설정:
- self_play_league에 deterministic `safe_heuristic` anchor를 0.15 weight로
  주입. 나머지 0.85는 current/recent/older self snapshot.
- traversal에서 15%의 opponent role을 휴리스틱이 담당 → self-mirror 평형
  부분 절단.

가설: "self-mirror over-opening 평형을 안chor가 깰 수 있는가."

결과 (1219 iter / 4h 풀 런):

| metric | 값 |
| --- | ---: |
| anchor traversal rate | 15.7% (mechanism 정상) |
| safe 상대 avg_diff | -57.11 |
| opened_colors | 4.83 |
| 5-color opening 빈도 | ~86% |
| random avg_diff | +48.31 (후퇴 없음) |

판정 (commit 메시지 그대로): "anchor pressure 0.15가 over-opening 평형을
깨지 못한 것으로 판정."

함의 — opponent pool에 외부 정책 일부 섞기는 0.15 정도로는 trap을 못 깬다.
더 큰 비율(0.5+)은 사실상 self-play 가설 폐기와 동일하므로 별개 카테고리로
취급해야 한다. discard_only를 anchor로 섞는 변형은 더 약한 신호이므로 같은
weight로는 더 나쁠 가능성이 큼.

### Negative unsampled regret was not sufficient

The `epsilon=0.05` plus `outcome_unsampled_regret=negative_node_value` variant
ended around safe strict score diff `-44.53` at 200 iterations, worse than
`epsilon=0.05` with `zero`.

Conclusion: the negative target suppresses opening, but it does not selectively
preserve good opens.

### The 200-iteration epsilon=0.05 peak did not hold

Confirmation run:

- Run: `runs/2026-05-09_010747_confirm-eps-005-zero-512x3-det-500`
- W&B group: `eps005-confirmation-512x3-v1`
- Config delta:
  - `run.deterministic=true`
  - `run.max_iterations=500`
  - `traversal.outcome_sampling_epsilon=0.05`
  - `traversal.outcome_unsampled_regret=zero`

Safe heuristic strict metrics:

| iteration | win rate | score diff | bad open rate | score/opened color |
| ---: | ---: | ---: | ---: | ---: |
| 200 | 0.12 | -40.01 | 0.893 | -6.25 |
| 250 | 0.08 | -46.91 | 0.900 | -7.29 |
| 300 | 0.08 | -45.93 | 0.895 | -6.30 |
| 350 | 0.07 | -54.02 | 0.899 | -7.83 |
| 400 | 0.04 | -49.64 | 0.912 | -7.83 |
| 450 | 0.06 | -49.31 | 0.917 | -6.71 |
| 500 | 0.06 | -61.24 | 0.903 | -8.30 |

Conclusion: `epsilon=0.05` creates a real short-run peak, but the behavior is
not stable through 500 iterations.

### LCFR time-weighting is not the sole cause of the degradation

Weighting ablation group:

- W&B group: `eps005-weighting-ablation-512x3-v1`
- Common config:
  - `run.deterministic=true`
  - `run.max_iterations=300`
  - `traversal.outcome_sampling_epsilon=0.05`
  - `traversal.outcome_unsampled_regret=zero`

Compared runs:

| run | iter 200 diff | iter 250 diff | iter 300 diff | iter 300 win | iter 300 bad open |
| --- | ---: | ---: | ---: | ---: | ---: |
| LCFR alpha=1.0 confirmation | -40.01 | -46.91 | -45.93 | 0.08 | 0.895 |
| `training_weighting.mode=none` | -50.98 | -48.46 | -56.27 | 0.04 | 0.912 |
| `training_weighting.lcfr_alpha=0.5` | -66.81 | -58.97 | -50.77 | 0.10 | 0.891 |

Open-quality comparison:

| run | iter 200 score/opened color | iter 300 score/opened color |
| --- | ---: | ---: |
| LCFR alpha=1.0 confirmation | -6.25 | -6.30 |
| `training_weighting.mode=none` | -7.71 | -7.70 |
| `training_weighting.lcfr_alpha=0.5` | -9.94 | -7.85 |

Conclusion: neither removing time-weighting nor softening LCFR to `alpha=0.5`
beat the original `alpha=1.0` confirmation by the primary score-diff metric at
300 iterations. LCFR may affect stability, but it is not the main lever.

## First-open diagnostic

Diagnostic output:

- `runs/tmp/first_open_advantage_confirm_eps005_200_vs_500.jsonl`

Observed values:

| checkpoint | bad selection | good selection | bad advantage | good advantage | sampled turns |
| --- | ---: | ---: | ---: | ---: | ---: |
| iter 200 | 0.0625 | 0.0582 | -12.82 | -16.60 | 8560 |
| iter 500 | 0.0067 | 0.0067 | -35.42 | -44.32 | 40942 |

Interpretation: by 500 iterations the model strongly suppresses opening
overall. It suppresses good opens along with bad opens, which is the core
selectivity failure.

## First-open target audit

Script:

- `scripts/analyze_first_open_targets.py`

Output:

- `runs/tmp/first_open_target_audit_confirm_eps005_200_vs_500.jsonl`

Method: regenerate short interleaved traversal batches from existing
checkpoints and bucket first-open advantage targets by action quality. Because
`outcome_unsampled_regret=zero` sets unsampled legal actions to zero, the most
informative statistic is the sampled-action target distribution, not the full
legal-candidate target distribution.

Sampled target summary:

| checkpoint | bucket | candidates | policy prob | sampled rate | sampled target mean | sampled target positive |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| iter 200 | good open | 796 | 0.078 | 0.078 | -40.71 | 0.419 |
| iter 200 | bad open | 7142 | 0.081 | 0.081 | -25.65 | 0.424 |
| iter 500 | good open | 800 | 0.064 | 0.058 | 8.39 | 0.435 |
| iter 500 | bad open | 8322 | 0.069 | 0.071 | 12.61 | 0.433 |

Interpretation: the regenerated traversal targets do not rank good opens above
bad opens. At both inspected checkpoints, bad-open candidates receive slightly
higher policy probability and sampled rate than good-open candidates. The
sampled target mean is also better for bad opens than good opens. This points
to a target or metric-alignment problem before model capacity or LCFR tuning.

## First-open counterfactual audit

Script:

- `scripts/analyze_first_open_counterfactual.py`

Output:

- `runs/tmp/first_open_counterfactual_confirm_eps005_200_vs_500.jsonl`

Method: collect first-open candidate states from existing checkpoints, force
each first-open candidate once, and compare the resulting continuation value
against the current policy's best non-open action from the same state.

`delta_open = value(force open) - value(best non-open)`.

Counterfactual summary against `heuristic_cautious`:

| checkpoint | bucket | candidates | delta mean | delta median | delta positive | policy prob | selected rate |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| iter 200 | good open | 40 | -26.57 | -26.5 | 0.050 | 0.000 | 0.000 |
| iter 200 | bad open | 460 | -23.15 | -21.0 | 0.130 | 0.036 | 0.037 |
| iter 500 | good open | 30 | -23.43 | -17.5 | 0.167 | 0.067 | 0.067 |
| iter 500 | bad open | 470 | -11.18 | -8.0 | 0.226 | 0.020 | 0.019 |

Interpretation: the heuristic `open_bad` label is not entirely misaligned with
continuation value. Forced bad opens are usually worse than the best non-open
alternative. However, heuristic `open_good` also often loses to best non-open in
these sampled states, so "recoverable eventually" is not the same as "open now."

Combined with the target audit, this points toward target/objective alignment:
the traversal target is not making the bad-open-vs-non-open mistake clearly
negative, even when the counterfactual continuation usually is negative.

## Open questions

1. Does the traversal target itself provide separable labels for good first
   opens versus bad first opens?
2. Is the model receiving too sparse or too noisy a signal at the first-open
   decision point?
3. Would target shaping around first-open decisions improve score/opened color
   without increasing bad-open rate?
4. Is evaluation showing a policy-selection problem, or is the advantage model
   already misranking good and bad opens before strategy extraction?

## Recommended next experiments

### 1. Deeper first-open target audit

Before another long training run, inspect sampled first-open decision records
more directly:

- Group candidate first-open actions into good-open and bad-open buckets.
- Compare target values before model prediction, not only final learned
  advantages.
- Report distributions, not just means.
- Check whether the target ranks good opens above bad opens in the same
  information-state context family.

Success criterion: the target distribution should show a usable separation
between good and bad opens. If it does not, the training target is the blocker.

### 2. First-open replay reweighting

Implemented option:

- `optimization.advantage_first_open_fraction`

Meaning: reserve this fraction of each advantage minibatch for samples where a
first-open action was legally available at the traverser decision. The target is
not changed; only replay sampling frequency changes. This keeps the experiment
on the pure self-play side more than hand-written target shaping.

Initial planned run:

- `run.experiment_name=first-open-reweight-50-512x3-det-500`
- `optimization.advantage_first_open_fraction=0.5`
- `traversal.outcome_sampling_epsilon=0.05`
- `traversal.outcome_unsampled_regret=zero`
- `run.deterministic=true`
- `run.max_iterations=500`

Primary comparison is the `eps=0.05` confirmation run and the pure external
sampling run. Success requires bad-open rate and score/opened color to improve
together without a large score-diff regression.

Result:

- Run: `runs/2026-05-09_235808_first-open-reweight-50-512x3-det-500-indexed`
- W&B group: `first-open-replay-v1`
- Commit: `8217cec` (`Speed up first-open memory sampling`)

The scan-based first implementation was stopped after 12 iterations because
first-open sampling scanned the full replay memory and pushed iteration time
above 60 seconds. The indexed-memory version kept first-open sampling near
0.25 seconds per player at 4M advantage samples and completed 500 iterations.

Final `heuristic_cautious` comparison:

| Run | Iter | Score diff | Win rate | Bad open | Score/opened |
| --- | ---: | ---: | ---: | ---: | ---: |
| baseline `confirm-eps-005-zero-512x3-det-500` | 500 | -61.24 | 0.06 | 0.903 | -8.30 |
| pure external `pure-external-512x3-det-500` | 500 | -50.38 | 0.06 | 0.925 | -9.79 |
| first-open reweight 50% indexed | 500 | -52.20 | 0.07 | 0.919 | -9.39 |

Conclusion: first-open replay reweighting alone did not solve selectivity. It
improved score diff versus the baseline final checkpoint, but it did not reduce
bad-open rate and made score/opened color worse. The run briefly looked better
around 120-200 iterations, then regressed by 300-500 iterations. This suggests
the replay emphasis is not enough if the underlying target does not separate
good first opens from bad first opens.

### 3. First-open target prior (A1) — running 2026-05-10

Hypothesis: target audit shows the regenerated traversal target does not
separate good first opens from bad first opens. With
`outcome_unsampled_regret=zero`, every unsampled first-open candidate is
labeled with target `0`, regardless of whether it is heuristically good
or bad. A small signed prior on unsampled first-open play actions should
break the symmetry without overriding the high-variance sampled-action
target.

Implementation:

- Config field: `traversal.outcome_unsampled_first_open_prior_alpha` (float,
  default `0.0`).
- Plumbed through `InterleavedTraversalConfig` and
  `run_interleaved_traversal_batch`. Cython traversal path unchanged
  (default scheduler is interleaved; pyx path is a follow-up if needed).
- Effect site: `_after_child` in `interleaved_traversal.py`. When
  `is_first_open` and `alpha != 0`, unsampled legal play actions whose color
  has an empty expedition are overwritten in `target` with `+alpha` if the
  visible `recoverable_score` for that color is `>= 0`, else `-alpha`. The
  sampled action's cell stays as `sampled_action_value - node_value`.
- `recoverable_score` mirrors `evaluate._visible_recoverable_summary` for
  the empty-expedition case (no future-card lookahead, hand-only signal).
- Tests: `test_first_open_prior_overrides_unsampled_play_targets_with_signed_alpha`,
  `test_first_open_prior_zero_alpha_is_noop`.

Note on "pure self-play": this prior introduces a hand-written heuristic
into the advantage target. The opponent and rollout policy remain
self-play (no external bot). This is treated as a diagnostic experiment;
if effective, the permanent solution is a counterfactual-based prior
(A2) that recovers full pure self-play.

Initial planned run:

- `run.experiment_name=first-open-prior-alpha5-512x3-det-200`
- `traversal.outcome_unsampled_first_open_prior_alpha=5.0`
- `traversal.outcome_unsampled_regret=zero`
- `traversal.outcome_sampling_epsilon=0.05`
- `run.deterministic=true`
- `run.max_iterations=200`
- W&B group: `first-open-prior-v1`

Primary comparisons:

- `confirm-eps-005-zero-512x3-det-500` (baseline; iter 200 reference)
- `first-open-reweight-50-512x3-det-500-indexed` (replay-side intervention)

Success criterion: `bad_open_rate` and `score_per_opened_color` improve
*together* at iter 200 without large regression in `avg_score_diff0`. If
positive, follow up with a 500-iter run to test stability. If null/regress,
try alpha sweep (e.g. 2.0, 10.0) before abandoning the direction.

Result (2026-05-10):

- Run: `runs/2026-05-10_072718_first-open-prior-alpha5-512x3-det-200`
- W&B: synced online to group `first-open-prior-v1` (run `teuh915r`)
- Commit: see HEAD at run start

`heuristic_cautious` at iter 200:

| Run | Score diff | Win rate | Bad open | Score/opened |
| --- | ---: | ---: | ---: | ---: |
| baseline `confirm-eps-005-zero` (iter 200) | -40.01 | 0.12 | 0.893 | -6.25 |
| **A1 prior α=5.0 (iter 200)** | **-67.54** | **0.02** | **0.796** | **-8.85** |

Other opponents at iter 200: random +32.51 / 0.86, heuristic_noisy -71.52 / 0.07,
heuristic_balanced -81.81 / 0.02, heuristic_aggressive -81.24 / 0.05.

Conclusion: **mixed result, net regression.** The prior did shift behavior
in the intended direction on one axis — `bad_open_rate` dropped from 0.893
to 0.796 (~10% absolute reduction). This is the only metric where the
hypothesis "the prior breaks ranking symmetry" looks supported.

But the overall game-quality metrics regressed: score diff worsened
(-40 → -67), win rate collapsed (0.12 → 0.02), and `score_per_opened_color`
got worse (-6.25 → -8.85). The model fails the success criterion (which
required `bad_open_rate` AND `score_per_opened_color` to improve together).

Two interpretations:

1. **α = 5.0 too strong.** The prior is overpowering the sampled-action
   target rather than acting as a weak symmetry-breaker. The good-open prior
   pushes the model to open *more often*, but those forced opens are bad
   *given the actual game state*, just labeled good by the visible-only
   heuristic. The counterfactual audit already flagged this: heuristic
   `open_good` candidates often lose to best non-open in real continuation.
2. **Heuristic itself misaligned.** Even the right α won't help if the
   sign assigned to candidates is wrong relative to true game value.

Next steps (in order):

- (A1.b) α sweep at 200 iter: α ∈ {1.0, 2.0} to test "weaker prior" hypothesis.
  If α=2.0 still regresses score diff while reducing bad_open, the prior shape
  itself is wrong, not just strength.
- (A2) Counterfactual prior: replace heuristic sign with sign of
  `value(force open) - value(best non-open)` from a small in-traversal
  rollout. Cleaner signal, recovers pure self-play, but more expensive.
- If both fail to improve score diff, the issue is upstream of unsampled
  regret labeling (likely traversal sample distribution or sampled-action
  target variance), and the next experiment family should target those.

### 4. D1 diagnostic: counterfactual with strong post-policy (2026-05-10)

Question: are forced-open continuation values low because the openings
themselves are bad, or because the self-play rollout policy poisons the
post-action play (selection bias)?

Method: re-ran `analyze_first_open_counterfactual.py` on the
`confirm-eps-005-zero-512x3-det-500` baseline checkpoints (iter 200, iter
500), but with `--post-policy heuristic_cautious`. The opponent and
state-collection policy stayed the same; only the policy_player's actions
*after* the forced first action used the strong fixed bot.

Output: `runs/tmp/first_open_counterfactual_d1_strong_post_policy_200_vs_500.jsonl`

`delta_open = value(force open) - value(best non-open)` comparison:

| iter | bucket | n | self-play post | strong post | shift |
| ---: | --- | ---: | ---: | ---: | ---: |
| 200 | open_good | 40 | -26.57 | **-3.35** | +23.22 |
| 200 | open_bad | 460 | -23.15 | **-5.52** | +17.63 |
| 500 | open_good | 30 | -23.43 | **-10.90** | +12.53 |
| 500 | open_bad | 470 | -11.18 | **-8.99** | +2.19 |

`delta_positive_rate`:

| iter | bucket | self-play post | strong post |
| ---: | --- | ---: | ---: |
| 200 | open_good | 0.050 | 0.225 |
| 200 | open_bad | 0.130 | 0.317 |
| 500 | open_good | 0.167 | 0.300 |
| 500 | open_bad | 0.226 | 0.230 |

Verdict — two findings, both important:

**1. Selection bias is real and significant.** Strong post-policy
improves forced-open continuation values by 12–23 points on average. The
self-play policy is meaningfully poisoning rollouts: forced opens look
much less bad once a competent player handles the followup.

**2. Heuristic labels do not separate cleanly even under strong post-policy.**
At iter 200, `open_good` delta_mean is only ~2 points better than
`open_bad` (-3.35 vs -5.52). At iter 500, ranking is essentially flat
(`open_good` -10.9 vs `open_bad` -8.99 — slightly *worse*). Median deltas
agree. Sample size for `open_good` is small (30–40) so noise contributes,
but there is no clean signal that the heuristic `recoverable_score`
classifier matches actual continuation value.

Implications:

- A1 prior was destined to fail — the heuristic sign is at best weakly
  aligned with continuation value, even with optimistic post-policy.
- A2 with self-play rollouts would inherit selection bias and likely
  reproduce the same misranking. A2 with a strong post-policy would
  give clean signs but breaks pure self-play.
- The deeper bottleneck is post-open play quality. Until the trained
  policy plays competently *after* opening, training signals about
  *whether to open* will be biased toward "don't open."

Candidate next directions (decision pending):

- (E1) Train with `cutoff_rollout_policy=heuristic_balanced` instead of `random`.
  Already a config option; gives leaf nodes stronger value estimates
  during traversal. Trades some pure-self-play purity for a stronger
  bootstrap signal. Cheap to test.
- (E2) Investigate post-open behavior directly: forced-open + observe
  next-2-3 turns. Diagnoses *why* the model can't follow up (e.g. always
  discards followup cards, switches color, etc.).
- (E3) Curriculum / staged training that exposes the network to good
  post-open trajectories before forcing it to make first-open decisions.
- A2 deferred unless we adopt a strong post-policy in rollouts.

### 5. E2 diagnostic: post-forced-open behavior (2026-05-10)

Question: D1 showed selection bias is real — *what is the model actually
doing after a forced first-open* that poisons rollouts?

Method: forced each first-open candidate at sampled states, then observed
the policy_player's next 3 decisions (window). Used baseline checkpoints
(`confirm-eps-005-zero` iter 200, iter 500). Categorized each followup
decision relative to the forced-open color.

Output: `runs/tmp/first_open_followup_baseline_200_vs_500.jsonl`
Script: `scripts/analyze_first_open_followup.py`

Per-window mean counts (3 policy_player decisions = ~1.5 game turns;
each game turn includes one play/discard plus one draw):

| iter | bucket | n | held@force | plays(same) | discards(same) | open_other | other_discard | held@end |
| ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 200 | good | 33 | 4.82 | 0.15 | 0.15 | **0.49** | 0.21 | 2.70 |
| 200 | bad  | 367 | 1.97 | 0.03 | 0.12 | 0.10 | 0.75 | 1.08 |
| 500 | good | 22 | 4.45 | 0.14 | **0.46** | 0.14 | 0.27 | 1.23 |
| 500 | bad  | 378 | 2.56 | 0.01 | 0.23 | 0.05 | 0.71 | 1.29 |

Findings:

**1. Model under-plays followup cards even when it holds many.** At iter
200, good-open candidates start with ~4.8 same-color cards in hand. In the
next 3 decisions, the model plays only 0.15 of them on average (3% of
its window). 2.7 cards remain in hand at terminal — nearly 3 cards of
the just-opened color never reach the expedition.

**2. Model opens additional new colors after a forced open.** At iter 200
good bucket, `open_other` = 0.49 in a 3-decision window — the model is
~3× more likely to open *another* new color than to follow up the one it
was forced into. This compounds the recoverable-score drag.

**3. By iter 500 the model actively dumps the forced color.** Good bucket
at iter 500: `same-color discards` 0.46 vs `same-color plays` 0.14. The
model discards followup cards more than 3× as often as it plays them,
despite holding 4.45 of them at force time. This is the strongest single
piece of evidence so far that the post-open value head is poisoned.

**4. Bad-open bucket shows even sharper avoidance.** Hand has ~2
same-color cards, plays effectively zero. Almost all play-phase decisions
are "discard other" (0.71–0.75). The model treats forced opens as bad
news to liquidate rather than commit to.

Interpretation: the trained advantage network *learned* that opens of
these colors are net-negative, so once forced into one, it hedges by
opening other colors and dumping followup cards. That hedging is rational
under the policy's own value estimates but is the exact mechanism that
keeps forced-open continuation values low and keeps the training target
labeling opens as bad. Closed loop.

This combined with D1 means:

- A1/A2 priors operate on first-open *labels*. They cannot fix a model
  that, even after committing to an open, refuses to follow up.
- Any fix needs to either (a) produce stronger leaf values during training
  so the value head learns post-open play matters, or (b) explicitly
  curriculum or prior-shape the *post-open* decision (not the open
  decision).

Updated next-step priorities:

- **(E1) `cutoff_rollout_policy=heuristic_balanced` training ablation.**
  Strongest single lever: gives traversal leaves stronger value estimates
  during training, which should ripple back to "open + follow up" signal.
  Pure-self-play purity dented, but only at cutoff leaves.
- (E1.b) Sanity check: at iter 200 baseline, the same-color discard rate
  is already 0.12 in good bucket — early. So this is not a late-training
  collapse; it's baked in from early iterations. Curriculum-style fix
  would have to start very early.
- A2 effectively dead unless paired with strong post-policy in rollouts.

### 6. Input feature audit and cleanup (2026-05-10)

After D1 + E2 confirmed the bottleneck is *training signal* and not *input
poverty*, audited every feature the model receives. Goal: align the input
representation with a strict pure-self-play definition by removing any
feature that embeds a hand-coded judgment or strategy assumption.

Three tiers found:

- **Tier 1**: pure game state and single-step game rules (hand cards,
  expedition state, discard piles, scores, public histogram, legal-action
  mask, `playable_*`/`dead_*` rule checks, `unknown_remaining_count`,
  etc.). No judgment. Kept.
- **Tier 2**: projection-based features that compute "if I commit and
  play all currently-playable cards, what is the score?" — mechanical
  but assumption-laden. Includes `recoverable_score_no_bonus`,
  `recoverable_margin_no_bonus`, `min_needed_to_break_even`,
  `cards_needed_for_bonus`, `has_bonus_path`. Removed.
- **Tier 3**: explicit hand-coded judgments — `is_bad_open_candidate`,
  `open_risk_score`, `is_safe_continuation`. Same heuristic family used
  to label `bad_open` in evaluation. Removed.

Notable additional finding: the `no_bonus` projection family is
asymmetric. It includes wager multipliers but excludes the +20 bonus,
which systematically *under*-estimates the upside of commitable
expeditions by roughly 20 × P(complete bonus). This bias points in the
same direction as the phase 1 trap ("opens look like loss"), so removing
the projection family is consistent with diagnosing-not-baking-in the
trap.

Implementation:

- `encoding.pyx`: `DERIVED_PLAYABILITY_PER_COLOR` 19 → 15;
  `SLOT_AWARE_PLAYABILITY_PER_SLOT` 12 → 6 (across two cleanup passes).
- Test shape assertions updated.
- Cython module rebuilt.

Input dimension change: 365 → 341 (Tier 3 removal) → **297** (Tier 2
removal). Net 18.6% reduction.

Result is not yet measured. The hypothesis is that with the heuristic
crutches removed, training behavior is a cleaner measurement of what
Deep CFR can do in this game from raw input. The model may stay in the
phase 1 trap longer or fail more visibly, both of which are useful
information.

### 7. ColorSharedNetwork chunked-layout bug (2026-05-10)

While re-examining the archived `2026-05-07_092137_color_shared_attention_1000iter`
run (killed at iter 41), discovered the `ColorSharedNetwork` implementation
in `networks.py` was not actually color-aware. The forward pass split the
input vector into `input_dim // n_colors` contiguous slices and ran them
through a shared encoder. The slice boundaries do not align with the actual
encoding layout — adjacent slices contain phase flags, hand slots,
expedition state, scores, etc. mixed together. The "shared color encoder"
was therefore sharing weights across semantically unrelated chunks, not
across per-color blocks.

This means the prior conclusion that "color_shared / attention archive run
was inconclusive" was charitable. The architecture being measured was a
chunked-input network mislabeled `color_shared`, not a real per-color
shared architecture. We have *no* signal on whether a properly per-color
architecture would help.

Fix landed in this same session:

- Added `compute_lost_cities_color_layout(input_dim)` in `networks.py`. For
  the standard Lost Cities schema (n_colors=5, hand_size=8, n_ranks=9), it
  recognises the four valid `input_dim` values (171, 219, 249, 297 across
  derived/slot-aware flag combinations) and returns per-color and common
  index lists derived from the actual encoding layout.
- Per-color block (39 dims with `derived_playability` on): both players'
  expedition state for that color, discard top metadata, public-histogram
  row, pending-discard one-hot bit, legal-action draw-pile bit, and the
  `derived_playability` per-color block. Slot-aware features stay in
  common because they are slot-major, not color-major.
- `ColorSharedNetwork.forward` now indexes per-color blocks via the layout
  when `input_dim` matches a known schema. For other input dims (unit
  tests, non-Lost Cities use), it falls back to the old chunked slicing
  with a `UserWarning`, preserving backward compatibility for tests but
  making the legacy behaviour visible.
- New unit tests cover both branches and the layout helper.

This is purely an implementation correctness fix; no fair test of the
architecture has been run yet. Fair test deferred — the diagnosis from
sections 4–5 (selection bias, post-open behaviour) suggests that even a
correct color-aware encoder would not break the closed loop on its own.

### 8. Interleaved scheduler honours all_negative_fallback (2026-05-10)

The interleaved traversal scheduler's `_regret_matching` was hard-coded to
spread fallback policy uniformly across legal actions, regardless of the
configured `regret_matching.all_negative_fallback`. The default config has
shipped with `all_negative_fallback: argmax_tiebreak` since
`618d5f8 Promote avg-strategy 1000iter to default.yaml` based on prior
20-iteration audit + 1000-iteration empirical evidence (see
`docs/archive/deep-cfr-regret-fallback-audit-2026-05-07.md`), but the
default scheduler was switched to interleaved in `09bbe7c Make interleaved
traversal the default`, after which the configured fallback mode silently
no-op'd in interleaved code paths.

Fix:

- `_regret_matching(advantages, legal_mask, epsilon, fallback_mode="uniform")`
  in `interleaved_traversal.py` now honours `argmax_tiebreak` by
  concentrating policy mass on the lowest-index tied action (deterministic
  tiebreak; the Cython recursive traverser randomises ties using its
  per-traverser RNG, which the batched policy does not have).
- `BatchedPolicy` accepts `fallback_mode` and threads it through.
- `InterleavedTraversalConfig` carries `all_negative_fallback`.
- `run_interleaved_traversal_batch`, `trainer.py`, `workers.py`, and the
  `analyze_first_open_targets.py` callers pass the field through.

Also bumped `traversal.outcome_sampling_epsilon` in `default.yaml` from
0.2 to 0.05. The 200-iteration sweep (section 1) showed 0.05 produced the
best short-run heuristic_cautious score diff (-40.01 vs -57.87 for
0.20). All recent experimental runs already used 0.05; the default now
matches actual experimental practice.

These changes do not target the diagnosed selection-bias bottleneck. They
align config intent with actual scheduler behaviour and make the default
config reproduce known-best knob settings out of the box.

### 9. Naming, plot curation, and tiered eval cadence (2026-05-10)

Hygiene changes — none target the diagnosed selection-bias bottleneck,
but they make the codebase honestly reflect the pure-self-play stance
and reduce dashboard noise.

Bot family rename (drop the unhelpful `safe_` prefix; suffixes now
describe behaviour):

| Old | New |
| --- | --- |
| `safe_heuristic_loose` | `heuristic_aggressive` |
| `safe_heuristic` | `heuristic_balanced` |
| `safe_heuristic_strict` | `heuristic_cautious` |
| `noisy_safe` | `heuristic_noisy` |
| `passive_discard` | `discard_only` |

Class renames in `bots/`: `SafeHeuristicBot` → `HeuristicBot`,
`SafeHeuristicParams` → `HeuristicParams`, `PassiveDiscardBot` →
`DiscardOnlyBot`, plus the loose/strict parameter constants. Backwards
compatibility was dropped intentionally — no aliases. Active configs,
docs, scripts, tests updated; archive files (read-only by policy)
left intact and may still reference old names.

Analyze plot curation (`deep_cfr/analyze.py`):

- New `analysis_00_core.png` dashboard with 10 heuristic-free metrics
  (loss/{advantage, strategy}; vs `heuristic_cautious`:
  `avg_score_diff0`, `win_rate0`, `avg_opened_colors`,
  `positive_expedition_rate`, `bonus_expedition_rate`,
  `score_per_opened_color`, `policy_entropy`; vs `random`: `win_rate0`).
- Removed `analysis_05_open_quality.png` (bad/weak/good open rates,
  recoverable score) and `analysis_07_calibration.png` (calibration gap,
  recoverable mean) — both relied on the heuristic `recoverable_score`
  classifier we already dropped from inputs.
- Removed `SELECTIVITY_PLOTS` and `plot_selectivity` (heuristic-laden).
- `SUMMARY_EVAL_METRICS` no longer includes `bad_open_rate` or
  `calibration_gap`.

`PlotSpec` gained an optional `opponents` allowlist so the new core
section can pin a specific opponent per panel without restructuring the
existing `plot_section` plumbing.

Tiered evaluation cadence (`EvaluationConfig`):

- Added `extended_opponents: tuple[str, ...]` and `extended_eval_every:
  int = 0`.
- Method `opponents_for_iteration(iteration)` returns the core list at
  every `eval_every`, and appends `extended_opponents` (de-duplicated)
  when `iteration` is also a multiple of `extended_eval_every`.
- `default.yaml` now uses 3 core opponents
  (`random`, `discard_only`, `heuristic_cautious`) every 5 iterations
  and 3 extended opponents (`heuristic_balanced`, `heuristic_aggressive`,
  `heuristic_noisy`) every 50 iterations.
- `random` is the floor sanity. `discard_only` is the zero-pit
  detector / absolute-score reference (its score is always 0, so
  `eval/discard_only/avg_score_diff0` directly equals our model's
  raw average score). `heuristic_cautious` is the ceiling and the
  archive-comparable benchmark used in sections 1–6.

Net effect on ongoing eval cost: ~50% reduction (3 opponents × every
5 iter, plus 6 opponents × every 50 iter, vs the prior 6 opponents
× every 5).

### 10. Short open-selectivity ablation

Run a 200-300 iteration ablation only after the target audit identifies a
specific change. Candidate changes include:

- first-open target shaping,
- modified unsampled-open penalty,
- outcome sampling focused on open-relevant branches,
- or strategy/eval selection that separates current and average policy at the
  open decision.

Primary metrics:

- `eval/heuristic_cautious/avg_score_diff0`
- `eval/heuristic_cautious/win_rate0`
- `eval/heuristic_cautious/bad_open_rate`
- `eval/heuristic_cautious/score_per_opened_color`

Do not promote to 500+ iterations unless bad-open rate and score/opened color
both improve without degrading score diff.

## Operational notes

- Keep one GPU training run active at a time.
- Use `tmux` plus `.compute.lock` for training.
- Mirror real experiments to W&B.
- Use one W&B group per hypothesis family.
- Do not start a long 2000-iteration run until a short diagnostic run shows
  stable selectivity improvement.

## §10. R1 — discard_only opponent diagnostic (2026-05-10)

### Hypothesis

Trap의 본질이 **자기참조 회로(self-similar opponent)**인지 **모델 자체
credit assignment 실패**인지 분리하기 위해, opponent_policy를
`discard_only` (항상 카드만 버리는 fixed bot)로 고정. opponent
dynamics를 거의 0으로 만들고 self-reference를 끊은 환경에서 모델이
trap을 벗어나는지 본다.

### Run

- Run dir: `runs/2026-05-10_173912_r1-discard-only-opp-512x3-det-300`
- W&B group: `r1-discard-only-opp-v1`
- Iters: 300 (R0 끊긴 시점과 같은 길이로 비교)
- Single change vs R0: `traversal.opponent_policy: discard_only`
- All else (capacity 2M, eval games 100, eps=0.05) identical

### Result (iter 300, vs R0 clean-baseline iter 300)

| metric | R0 (over-open) | R1 (zero-pit) | Δ |
| --- | ---: | ---: | ---: |
| score_diff vs cautious | -47.99 | **-69.56** | -21.57 worse |
| opened_colors vs cautious | 4.93 | **2.54** | -2.39 |
| play_action_rate vs cautious | 0.02 | **0.00** | -0.02 |
| pos_expeditions/game cautious | 0.76 | **0.08** | -0.68 |
| neg_expeditions/game cautious | 4.06 | 2.44 | -1.62 |
| avg_game_length cautious | 1053 | **5282** | +4229 (timeout) |
| score_diff vs discard_only | -27.19 | **0.00** | +27.19 |
| opened_colors vs discard_only | 4.98 | **0.00** | -4.98 |
| bad_open_rate vs discard_only | 0.91 | **0.00** | -0.91 |
| score_diff vs random | +49.61 | **-9.30** | -58.91 |
| opened_colors vs random | 4.97 | 1.45 | -3.52 |

### Interpretation: trap shifted, did not break

Trap 본질 진단 결과 **"자기참조 회로"는 충분조건이 아니다.**

- vs `discard_only` opponent: 모델이 **정확히 0개 색을 열고 0점으로 비김**
  (zero-pit). 즉 closed-loop self-reference 깬 후에도 trap이 다른
  형태(zero-pit)로 재출현.
- vs `random`: R0에서 +49 (제압)하던 상대에게 R1은 -9 (패배). random은
  카드를 두기라도 하니까 음수 expedition으로 끝나도 양수 게임 일부 만들어서 점수
  내지만, R1 모델은 0점 (전혀 안 둠).
- vs `heuristic_cautious`: opened_colors 절반(4.93→2.54)으로 줄었지만
  play_action_rate 0% — 연 expedition을 채우지 못함. avg_game_length
  5x 증가 → max_steps timeout 다발. score_diff 22점 더 악화.
- Action 분포 (vs cautious): **discard 50% / draw_pile 50% / play 0% /
  draw_deck 0%**. 즉 "버리고 상대 버린 거 줍기"의 단일 패턴.
- Loss 트렌드: advantage loss 1500 → 3500 (꾸준히 상승), strategy loss
  0.7 → 1.4 (꾸준히 상승). 모델이 점점 더 noisy한 target을 못 fit함.

### What this proves

1. **Trap의 본질은 모델 자체 value function 실패 (credit assignment)**, not
   self-reference. self-reference는 trap 강화 요인이지만 제거해도 다른 형태로
   재출현.
2. **모델이 일관되게 못하는 일은 "expedition에 카드 두기" 행동**. R0에서
   over-open 무차별 5색 + 채우지 못함. R1에서 아예 안 열되 채우지도 않음.
   양 극단 어디든 play_action_rate ≈ 0%.
3. **Outcome sampling + 본인 정책 rollout만으로 long-horizon credit chain을
   못 닫는다.** "open이 좋은가?"의 답은 "이후 카드를 잘 두는가"에 종속이고,
   모델이 카드 두기를 못 배우는 한 open 가치는 노이즈로만 추정됨.

### Recommended next direction

이전에 hygiene 개선(R1 = 4M capacity + 500 eval)을 첫 후보로 두었으나, R1의
명확한 결과로 hygiene으로는 trap 본질 못 풀 것이 거의 확정. **R2는 trap에 직접
작용하는 변경이어야 함.**

후보 (강도 약→강):

1. **Curriculum (작은 게임 → 큰 게임)**: 3색 5랭크 mini Lost Cities로 시작 →
   credit chain 짧아짐 → expedition 사이클이 게임 8턴 안에 닫힘. 모델이
   "카드 두기" 학습 가능성 확보. 풀 게임에 weight transfer.
2. **Opening / playing 네트워크 분리**: closed loop 회로의 한 회선 물리적
   절단. opening 네트워크는 follow-up이 noise여도 자체 신호로 학습 가능.
   아키텍처 작업 큼.
3. **Cutoff rollout heuristic + external opponent (조합)**: opening 가치
   추정에 외부 합리적 정책 끼움. pure self-play 가설 명확히 폐기.

저자 추천: **R2 = curriculum**. 가장 적은 아키텍처 변경으로 credit chain
직접 단축. 작은 게임에서 follow-up 학습 성공이 풀 게임 학습의 핵심 prior가
됨.


## §11. R2 — heuristic_balanced opponent diagnostic (2026-05-10)

### Hypothesis

R1에서 trap의 본질이 자기참조 회로 아님이 확정됐지만 R1은 discard_only 상대라
zero-pit이 *수학적으로 정답*이라 진단이 흐릿했다. R2는 양수 점수 만드는
competent 상대(heuristic_balanced)를 두면 zero-pit 균형이 사라지고 모델이
"카드 두기" 학습 신호를 처음으로 분명히 받는지 확인.

### Run

- Run dir: `runs/2026-05-10_211554_r2-heuristic-balanced-opp-512x3-det-300`
- W&B group: `r2-heuristic-balanced-opp-v1`
- Config: `traversal.opponent_policy: heuristic_balanced` (기본 self_play_league
  → 휴리스틱). 다른 모든 변수 R0/R1과 동일.
- Iters: 167 / 300 종료 (조기 중단). 패턴 명확히 드러나서 끝까지 갈 가치 없음.
- 부수: interleaved scheduler에 heuristic_balanced opponent 새로 추가
  (commit `7d59398`), recursive 안 쓰고 빠른 path 유지.

### Result (last eval iter 165)

| metric | R0 | R1 | R2 |
| --- | ---: | ---: | ---: |
| vs cautious score_diff | -47.99 | -69.56 | -54.38 |
| vs cautious opened_colors | 4.93 | 2.54 | 1.11 |
| **vs cautious play_action_rate** | **0.02** | **0.00** | **0.00** |
| vs cautious per_game_pos_exp | 0.76 | 0.08 | 0.02 |
| vs cautious per_game_neg_exp | 4.06 | 2.44 | 1.08 |
| vs cautious avg_game_length | 1053 | 5282 | 7622 |
| vs cautious bad_open_rate | 0.92 | 0.96 | 0.98 |
| vs random score_diff | +49.61 | -9.30 | +5.07 |
| vs discard_only score_diff | -27.19 | 0.00 | -21.54 |
| advantage loss (steady) | ~2300 | 1500→3500 | ~200 |

### Interpretation: trap shape changes, root cause invariant

R2는 R1의 zero-pit과 R0의 over-open 사이의 **변형**:
- opened_colors 1.11 (R0 5.0보다 작고, R1 0.0보다 큼)
- bad_open_rate **0.98** (R0/R1보다 더 나쁨) — 여는 거의 모든 색이 bad open
- positive expeditions **0.02개/게임** — 사실상 보너스 절대 못 받음
- avg_game_length 7622 — max_steps timeout 폭증, 게임 종료 못 시키는 패턴
- vs random에서 R0의 +50점 우위가 +5로 무너짐 → random에게도 거의 못 이김

R2의 advantage loss 200은 R0의 2200 대비 1/10. **모델이 trap 정책을 더
정확하게 학습 중**(targets가 deterministic heuristic 상대라 분산 작음).
정확한 fit이지만 fit하는 게 trap.

### What three runs together prove

| | R0 (self) | R1 (discard) | R2 (heuristic) |
| --- | ---: | ---: | ---: |
| opponent type | self-play | passive fixed | competent fixed |
| trap shape | over-open | zero-pit | small over-open + zero followup |
| **play_action_rate** | **~0%** | **~0%** | **~0%** |

**세 가지 다른 opponent 분포에서 모델 모두 `play_action_rate ≈ 0%`.** opponent
와 무관한 핵심 결함. credit assignment / 알고리즘 한계의 직접 증거.

### Closed: opponent 변경으로 trap 못 풂

이 시점에서 opponent dimension은 닫힘. R0(self), R1(passive), R2(competent)
세 카테고리 모두 trap. opponent 환경 변경은 trap의 외형만 바꾸고 본질을 못 풂.

### Recommended direction: SO-ISMCTS 정공법

이전 §10 결론에서 curriculum을 1순위로 두었으나 R2 결과로 우선순위 조정:

- credit assignment가 알고리즘 자체의 한계임이 R0/R1/R2 누적 증거로 거의 확정
- opponent 변형은 닫힌 카테고리
- 다음 정공법은 **search-based bootstrap (SO-ISMCTS)** — value head로 long-horizon credit chain 단축, PUCT search로 가치 추정 분산 줄임. AlphaZero가 비슷한 구조 풀어본 검증된 메커니즘.
- Curriculum과 network split은 같은 self-play 가족이지만 search 추가만큼 본질 공격은 아님 → R3는 SO-ISMCTS PoC, curriculum은 백업.

설계 문서 작성 + 단계별 구현은 별도 plan으로 (추가 예정).


## §12. R3 — SO-ISMCTS mini Lost Cities PoC (2026-05-10)

### Hypothesis

R0/R1/R2 누적 진단 결과 trap 본질이 알고리즘 자체의 long-horizon credit
assignment 실패임이 확정. 다음 정공법으로 SO-ISMCTS (Single-Observer
Information Set MCTS) + AlphaZero-style policy/value head 도입. value head로
terminal까지 walk 안 하고 search 부트스트랩으로 가치 추정.

이번 실험은 **mini Lost Cities (3색 5랭크, hand 4, bonus 4)** 에서 알고리즘이
작동하는지 PoC 검증. mini로 시작한 이유: full game은 horizon 길어 학습 비용
큼, mini에서 알고리즘 작동 확인 후 scale-up.

### Design

- 변종: SO-ISMCTS (정보집합 노드 트리 + 시뮬마다 새 determinization)
- Determinization: uniform from unseen cards (Bayesian belief 미도입)
- Selection: PUCT (c_puct=1.5)
- Network: AlphaZero식 backbone (MLP 128×2 mini용) + policy head + tanh×100 value head
- Loss: MSE(value) + KL(policy)
- Tree: 매 결정마다 fresh (subtree 재사용 없음)
- Chance: closed-loop within sim, re-determinize across sims
- Pure self-play (양 플레이어 같은 네트워크)
- 50 simulations / decision (PoC, 향후 200+)

### Implementation

신규 모듈: `src/coolrl_lost_cities/games/classic/ismcts/`
- info_set.py, determinization.py, network.py, mcts.py, replay_buffer.py,
  self_play.py, trainer.py, config.py, cli.py
- CLI: `lost-cities-ismcts train --config configs/ismcts/{mini,default}.yaml`
- 단위 테스트 7개 (정보집합 카노니컬, determinization 정합성, 네트워크 forward,
  MCTS smoke, self-play 게임 종료, replay 버퍼)
- 기존 Deep CFR 코드 무수정 (encoding, evaluate만 재사용)

Codex commit: `e69f316`.

### Result (mini, 50 iter, 20 eval games per opponent)

| 상대 | score_diff | win_rate | play_action_rate |
| --- | ---: | ---: | ---: |
| random | **+33.1** | 0.90 | 0.13 |
| **heuristic_cautious** | **-5.7** | **0.45** | **0.22** |
| discard_only | -20.8 | 0.00 | 0.31 |

비교 (full game Deep CFR R0/R1/R2):

| metric | Deep CFR (full game) | SO-ISMCTS (mini game) |
| --- | ---: | ---: |
| score_diff vs cautious | -48 ~ -70 | **-5.7** |
| heuristic_cautious 승률 | 0.06 | **0.45** |
| play_action_rate vs cautious | 0~2% | **22%** |

### Interpretation: trap escaped on mini

**Deep CFR이 6주 동안 못 풀던 핵심 결함 (play_action_rate ≈ 0%)이 단번에
해소.** 모델이 처음으로 expedition에 카드를 의미 있게 두고 있음 (13~31%).
heuristic_cautious 상대 45% 승률은 거의 호각 (mini라 quality threshold 다르긴
하지만 명확히 다른 학습 regime).

진단 가치: **"trap = long-horizon credit assignment, search 부트스트랩으로 풀
수 있음"** 가설 강하게 지지됨. mini는 horizon 짧아 (게임당 ~15-25 결정)
search bootstrap의 효과가 직접 확인 가능했음.

### Caveats

- Mini game은 credit chain이 짧음. full game (5색 9랭크, ~200 결정/게임)에선
  같은 효과 보인다는 보장 없음.
- Eval 20게임 small sample. SE 큼.
- 시뮬 수 50은 mini에는 충분했지만 full은 부족할 수 있음.
- Strategy fusion / sparse exploration 같은 SO-ISMCTS 자체의 알려진 한계는
  mini에서 노출 안 됨. full에선 노출될 가능성.

### Next: scale-up to full game

R3a 후보: full Lost Cities (5색 9랭크) 100~200 iter 학습. ETA 3~6시간 추정
(probe 측정 기반). configs/ismcts/default.yaml 기본 설정으로 시작.

성공 기준 (mini와 평행):
- vs random: score_diff > +30, win_rate > 0.7
- vs heuristic_cautious: play_action_rate > 10% 이상 (Deep CFR의 ~0%와 분명한 차이)
- 가능하면 score_diff > -30 (Deep CFR의 -50~-70보다 명확히 좋음)

