Strip heuristic input features and add selectivity diagnostics

Audit of the Deep CFR information-state encoding identified two tiers of
non-pure features and removed both:

- Tier 3 (judgment): is_bad_open_candidate, open_risk_score,
  is_safe_continuation. Same heuristic family used to label bad_open in
  evaluation, embedded as model input.
- Tier 2 (projection): recoverable_score_no_bonus,
  recoverable_margin_no_bonus, min_needed_to_break_even,
  cards_needed_for_bonus, has_bonus_path. Mechanical but assumption-laden
  ("commit and play all currently-playable cards"). The no_bonus form is
  asymmetric: it amplifies the immediate -20 penalty while truncating the
  +20 bonus upside, biasing the model toward the same "don't open" basin
  the diagnostics already flagged.

Input dim 365 -> 297. DERIVED_PLAYABILITY_PER_COLOR 19 -> 15;
SLOT_AWARE_PLAYABILITY_PER_SLOT 12 -> 6. Test shape assertions updated.

Also adds selectivity diagnostic infrastructure used to reach this point:
- traversal.outcome_unsampled_first_open_prior_alpha config field with
  signed-prior overlay on unsampled first-open advantage targets (A1).
- analyze_first_open_counterfactual.py --post-policy to swap the
  policy_player rollout policy and isolate selection bias (D1).
- analyze_first_open_followup.py to inspect post-forced-open behavior
  (E2): same-color play vs discard counts, other-open rate, terminal
  hand composition.

Findings recorded in docs/plans/deep-cfr-selectivity.md sections 3-6.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-10 14:26:16 +09:00
co-authored by Claude Opus 4.7
parent 1593135313
commit f63c4b8059
10 changed files with 786 additions and 37 deletions
+28 -2
View File
@@ -218,15 +218,27 @@ def _rollout_value(
opponent: str,
seed: int,
max_steps: int,
post_policy: str = "model",
) -> float:
"""Roll out from `state` to terminal and return policy_player's score diff.
When `post_policy == "model"`, policy_player uses the trained advantage
network policy. Otherwise `post_policy` is treated as a bot name and a
fresh bot is built for policy_player too — used to diagnose whether the
self-play rollout itself is poisoning forced-open continuation values.
"""
rollout = state.clone()
opponent_policy = build_bot(opponent, seed=seed)
post_policy_bot = build_bot(post_policy, seed=seed * 7 + 1) if post_policy != "model" else None
steps = 0
while not rollout.terminal and steps < max_steps:
current = int(rollout.current_player)
if current == policy_player:
unified = policy.select_unified(rollout)
action = rollout.from_unified_action(unified)
if post_policy_bot is not None:
action = post_policy_bot.act(rollout)
else:
unified = policy.select_unified(rollout)
action = rollout.from_unified_action(unified)
else:
action = opponent_policy.act(rollout)
rollout.apply_action(action)
@@ -260,6 +272,7 @@ def analyze_checkpoint(
device: torch.device,
max_steps: int,
max_candidates: int,
post_policy: str = "model",
) -> dict[str, Any]:
_cfg, game_config, policy, iteration = _load_checkpoint(checkpoint, device)
buckets: dict[str, Bucket] = defaultdict(Bucket)
@@ -306,6 +319,7 @@ def analyze_checkpoint(
opponent=opponent,
seed=game_seed * 10_000 + candidate_states * 101 + 1,
max_steps=max_steps,
post_policy=post_policy,
)
for open_action in open_actions:
if evaluated_open_candidates >= max_candidates:
@@ -319,6 +333,7 @@ def analyze_checkpoint(
opponent=opponent,
seed=game_seed * 10_000 + candidate_states * 101 + 2,
max_steps=max_steps,
post_policy=post_policy,
)
label = labels[open_action]
buckets[label].add(
@@ -344,6 +359,7 @@ def analyze_checkpoint(
"policy_turns": policy_turns,
"candidate_states": candidate_states,
"first_open_candidates": first_open_candidates,
"post_policy": post_policy,
"buckets": {key: bucket.to_dict() for key, bucket in sorted(buckets.items())},
}
@@ -357,6 +373,15 @@ def main() -> None:
parser.add_argument("--device", default="cuda")
parser.add_argument("--max-steps", type=int, default=10_000)
parser.add_argument("--max-candidates", type=int, default=500)
parser.add_argument(
"--post-policy",
default="model",
help=(
"Policy used for the policy_player during forced-action rollouts. "
"'model' uses the trained advantage network; any other value is "
"treated as a bot name (e.g. 'safe_heuristic_strict')."
),
)
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
@@ -372,6 +397,7 @@ def main() -> None:
device=device,
max_steps=args.max_steps,
max_candidates=args.max_candidates,
post_policy=args.post_policy,
)
rows.append(row)
print(json.dumps(row, sort_keys=True))