Rename bot family, curate analyze plots, tier evaluation cadence

Three coordinated hygiene changes; none target the diagnosed
selection-bias bottleneck. They make the codebase honestly reflect the
pure-self-play stance and reduce dashboard noise.

Bot rename (drop the unhelpful safe_ prefix; suffixes describe behaviour):
- safe_heuristic_loose -> heuristic_aggressive
- safe_heuristic       -> heuristic_balanced
- safe_heuristic_strict -> heuristic_cautious
- noisy_safe           -> heuristic_noisy
- passive_discard      -> discard_only

Class renames in bots/: SafeHeuristicBot -> HeuristicBot,
SafeHeuristicParams -> HeuristicParams, PassiveDiscardBot -> DiscardOnlyBot,
plus loose/strict parameter constants. Backwards compatibility was dropped
intentionally per user instruction; no aliases. Active configs, docs,
scripts, tests updated. Archive directories (configs/archive,
docs/archive, runs/archive) left intact and may still reference old
names per their read-only policy. The src/.../bots/passive.py module was
renamed to discard_only.py via git mv.

Analyze plot curation (deep_cfr/analyze.py):
- Added analysis_00_core.png as the canonical daily dashboard with 10
  heuristic-free metrics (loss/{advantage,strategy}; vs heuristic_cautious:
  avg_score_diff0, win_rate0, avg_opened_colors, positive_expedition_rate,
  bonus_expedition_rate, score_per_opened_color, policy_entropy; vs random:
  win_rate0).
- Removed analysis_05_open_quality.png (bad/weak/good open rates,
  recoverable score) and analysis_07_calibration.png (calibration gap,
  recoverable mean) - both relied on the heuristic recoverable_score
  classifier already dropped from inputs.
- Removed SELECTIVITY_PLOTS and plot_selectivity (heuristic-laden).
- SUMMARY_EVAL_METRICS no longer includes bad_open_rate or
  calibration_gap.
- PlotSpec gained an opponents allowlist so the new core section can pin
  a specific opponent per panel without restructuring plot_section.

Tiered evaluation cadence (EvaluationConfig):
- Added extended_opponents and extended_eval_every (default 0 = disabled).
- opponents_for_iteration(iteration) returns the core list every
  eval_every and appends extended_opponents (de-duplicated) when
  iteration is also a multiple of extended_eval_every.
- default.yaml now uses 3 core opponents (random, discard_only,
  heuristic_cautious) every 5 iterations and 3 extended opponents
  (heuristic_balanced, heuristic_aggressive, heuristic_noisy) every 50
  iterations. random is the floor sanity. discard_only is the
  zero-pit detector / absolute-score reference (its score is always 0,
  so eval/discard_only/avg_score_diff0 directly equals the model's raw
  average score). heuristic_cautious is the ceiling and the
  archive-comparable benchmark used in the prior diagnostic sections.

Net eval cost reduction: roughly 50% (3 opponents x every 5 iter, plus
6 opponents x every 50 iter, vs the prior 6 x every 5).

Documented in docs/plans/deep-cfr-selectivity.md section 9.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-10 15:32:55 +09:00
co-authored by Claude Opus 4.7
parent 0457efdf29
commit 004b913a7b
41 changed files with 441 additions and 324 deletions
+1 -1
View File
@@ -7,7 +7,7 @@ The current implementation starts with the classic two-player card game:
- classic 5-expedition rules by default - classic 5-expedition rules by default
- Python/Cython game engine - Python/Cython game engine
- env wrapper - env wrapper
- random, passive-discard, and safe-heuristic bots - random, discard-only, and safe-heuristic bots
- core rule, scoring, mask, env, canonical-state, bot, and GUI smoke tests - core rule, scoring, mask, env, canonical-state, bot, and GUI smoke tests
Training code, Deep CFR, learned-policy evaluation, GUI, and web client are Training code, Deep CFR, learned-policy evaluation, GUI, and web client are
+7 -5
View File
@@ -92,11 +92,13 @@ evaluation:
games: 100 games: 100
opponents: opponents:
- random - random
- passive_discard - discard_only
- safe_heuristic - heuristic_cautious
- safe_heuristic_loose extended_eval_every: 50
- safe_heuristic_strict extended_opponents:
- noisy_safe - heuristic_balanced
- heuristic_aggressive
- heuristic_noisy
max_steps: 10000 max_steps: 10000
on_max_steps: score_diff on_max_steps: score_diff
batch_size: 64 batch_size: 64
+5 -5
View File
@@ -89,11 +89,11 @@ evaluation:
games: 100 games: 100
opponents: opponents:
- random - random
- passive_discard - discard_only
- safe_heuristic - heuristic_balanced
- safe_heuristic_loose - heuristic_aggressive
- safe_heuristic_strict - heuristic_cautious
- noisy_safe - heuristic_noisy
max_steps: 10000 max_steps: 10000
on_max_steps: score_diff on_max_steps: score_diff
batch_size: 64 batch_size: 64
+10 -10
View File
@@ -182,7 +182,7 @@ eval_<opponent>_<metric>
For that run, iteration 75 had `evaluation_seconds = 18.29s`. Evaluation was For that run, iteration 75 had `evaluation_seconds = 18.29s`. Evaluation was
parallelized by opponent, so per-opponent `elapsed_seconds` values overlap and parallelized by opponent, so per-opponent `elapsed_seconds` values overlap and
must not be summed as wall-clock time. The slow safe-heuristic opponents must not be summed as wall-clock time. The slow heuristic opponents
dominated the eval wall-clock. dominated the eval wall-clock.
Representative per-opponent breakdown: Representative per-opponent breakdown:
@@ -190,13 +190,13 @@ Representative per-opponent breakdown:
| Opponent | Elapsed | Network | Postprocess | Opponent act | | Opponent | Elapsed | Network | Postprocess | Opponent act |
| --- | ---: | ---: | ---: | ---: | | --- | ---: | ---: | ---: | ---: |
| `random` | 0.57s | 0.18s | 0.25s | 0.06s | | `random` | 0.57s | 0.18s | 0.25s | 0.06s |
| `passive_discard` | 0.36s | 0.13s | 0.17s | 0.00s | | `discard_only` | 0.36s | 0.13s | 0.17s | 0.00s |
| `safe_heuristic` | 14.54s | 2.65s | 3.34s | 7.57s | | `heuristic_balanced` | 14.54s | 2.65s | 3.34s | 7.57s |
| `safe_heuristic_loose` | 11.20s | 2.55s | 3.30s | 4.63s | | `heuristic_aggressive` | 11.20s | 2.55s | 3.30s | 4.63s |
| `safe_heuristic_strict` | 15.94s | 2.51s | 3.14s | 9.22s | | `heuristic_cautious` | 15.94s | 2.51s | 3.14s | 9.22s |
| `noisy_safe` | 1.68s | 0.40s | 0.58s | 0.57s | | `heuristic_noisy` | 1.68s | 0.40s | 0.58s | 0.57s |
The important read is that safe-heuristic evaluation is not primarily GPU The important read is that heuristic evaluation is not primarily GPU
network forward time. `opponent_act_seconds` and policy post-processing are network forward time. `opponent_act_seconds` and policy post-processing are
larger than `policy_network_seconds` for the slowest opponents. larger than `policy_network_seconds` for the slowest opponents.
@@ -236,7 +236,7 @@ The practical eval tuning levers are:
this would require a feature change. this would require a feature change.
4. Reduce frequent opponents. 4. Reduce frequent opponents.
The safe-heuristic opponents dominate wall-clock in the inspected run. For The heuristic opponents dominate wall-clock in the inspected run. For
frequent checks, evaluate against one or two representative opponents and run frequent checks, evaluate against one or two representative opponents and run
the full suite less often. the full suite less often.
@@ -278,7 +278,7 @@ already implemented. It would replace or wrap the strategy-network forward pass
with a precompiled inference engine. Its maximum impact is bounded by with a precompiled inference engine. Its maximum impact is bounded by
`policy_network_seconds`, not by total eval time. `policy_network_seconds`, not by total eval time.
In the inspected eval row, the slow safe-heuristic opponents spent about In the inspected eval row, the slow heuristic opponents spent about
2.5-2.6s in policy-network forward but 4.6-9.2s in opponent action selection and 2.5-2.6s in policy-network forward but 4.6-9.2s in opponent action selection and
about 3.1-3.3s in policy post-processing. That means TensorRT could help eval, about 3.1-3.3s in policy post-processing. That means TensorRT could help eval,
especially for larger `evaluation.games`, but it is not expected to collapse the especially for larger `evaluation.games`, but it is not expected to collapse the
@@ -319,7 +319,7 @@ Based on the current metrics, the more plausible performance work is:
but it requires changing traversal scheduling, not just swapping the network but it requires changing traversal scheduling, not just swapping the network
backend. backend.
6. For eval-heavy runs, optimize the safe-heuristic opponents and policy 6. For eval-heavy runs, optimize the heuristic opponents and policy
post-processing before assuming TensorRT is the main lever. post-processing before assuming TensorRT is the main lever.
The inspected eval row shows those costs dominate the slowest opponents. The inspected eval row shows those costs dominate the slowest opponents.
+79 -15
View File
@@ -17,7 +17,7 @@ first-open advantage target.
## Baseline symptoms ## Baseline symptoms
The 512x3 dense-eval baseline showed improving training losses, but the main The 512x3 dense-eval baseline showed improving training losses, but the main
game-quality metrics against `safe_heuristic_strict` did not improve enough to game-quality metrics against `heuristic_cautious` did not improve enough to
indicate a useful policy. indicate a useful policy.
Observed pattern: Observed pattern:
@@ -176,7 +176,7 @@ against the current policy's best non-open action from the same state.
`delta_open = value(force open) - value(best non-open)`. `delta_open = value(force open) - value(best non-open)`.
Counterfactual summary against `safe_heuristic_strict`: Counterfactual summary against `heuristic_cautious`:
| checkpoint | bucket | candidates | delta mean | delta median | delta positive | policy prob | selected rate | | checkpoint | bucket | candidates | delta mean | delta median | delta positive | policy prob | selected rate |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
@@ -257,7 +257,7 @@ first-open sampling scanned the full replay memory and pushed iteration time
above 60 seconds. The indexed-memory version kept first-open sampling near above 60 seconds. The indexed-memory version kept first-open sampling near
0.25 seconds per player at 4M advantage samples and completed 500 iterations. 0.25 seconds per player at 4M advantage samples and completed 500 iterations.
Final `safe_heuristic_strict` comparison: Final `heuristic_cautious` comparison:
| Run | Iter | Score diff | Win rate | Bad open | Score/opened | | Run | Iter | Score diff | Win rate | Bad open | Score/opened |
| --- | ---: | ---: | ---: | ---: | ---: | | --- | ---: | ---: | ---: | ---: | ---: |
@@ -331,15 +331,15 @@ Result (2026-05-10):
- W&B: synced online to group `first-open-prior-v1` (run `teuh915r`) - W&B: synced online to group `first-open-prior-v1` (run `teuh915r`)
- Commit: see HEAD at run start - Commit: see HEAD at run start
`safe_heuristic_strict` at iter 200: `heuristic_cautious` at iter 200:
| Run | Score diff | Win rate | Bad open | Score/opened | | Run | Score diff | Win rate | Bad open | Score/opened |
| --- | ---: | ---: | ---: | ---: | | --- | ---: | ---: | ---: | ---: |
| baseline `confirm-eps-005-zero` (iter 200) | -40.01 | 0.12 | 0.893 | -6.25 | | baseline `confirm-eps-005-zero` (iter 200) | -40.01 | 0.12 | 0.893 | -6.25 |
| **A1 prior α=5.0 (iter 200)** | **-67.54** | **0.02** | **0.796** | **-8.85** | | **A1 prior α=5.0 (iter 200)** | **-67.54** | **0.02** | **0.796** | **-8.85** |
Other opponents at iter 200: random +32.51 / 0.86, noisy_safe -71.52 / 0.07, Other opponents at iter 200: random +32.51 / 0.86, heuristic_noisy -71.52 / 0.07,
safe_heuristic -81.81 / 0.02, safe_heuristic_loose -81.24 / 0.05. heuristic_balanced -81.81 / 0.02, heuristic_aggressive -81.24 / 0.05.
Conclusion: **mixed result, net regression.** The prior did shift behavior Conclusion: **mixed result, net regression.** The prior did shift behavior
in the intended direction on one axis — `bad_open_rate` dropped from 0.893 in the intended direction on one axis — `bad_open_rate` dropped from 0.893
@@ -382,7 +382,7 @@ post-action play (selection bias)?
Method: re-ran `analyze_first_open_counterfactual.py` on the Method: re-ran `analyze_first_open_counterfactual.py` on the
`confirm-eps-005-zero-512x3-det-500` baseline checkpoints (iter 200, iter `confirm-eps-005-zero-512x3-det-500` baseline checkpoints (iter 200, iter
500), but with `--post-policy safe_heuristic_strict`. The opponent and 500), but with `--post-policy heuristic_cautious`. The opponent and
state-collection policy stayed the same; only the policy_player's actions state-collection policy stayed the same; only the policy_player's actions
*after* the forced first action used the strong fixed bot. *after* the forced first action used the strong fixed bot.
@@ -434,7 +434,7 @@ Implications:
Candidate next directions (decision pending): Candidate next directions (decision pending):
- (E1) Train with `cutoff_rollout_policy=safe_heuristic` instead of `random`. - (E1) Train with `cutoff_rollout_policy=heuristic_balanced` instead of `random`.
Already a config option; gives leaf nodes stronger value estimates Already a config option; gives leaf nodes stronger value estimates
during traversal. Trades some pure-self-play purity for a stronger during traversal. Trades some pure-self-play purity for a stronger
bootstrap signal. Cheap to test. bootstrap signal. Cheap to test.
@@ -510,7 +510,7 @@ This combined with D1 means:
Updated next-step priorities: Updated next-step priorities:
- **(E1) `cutoff_rollout_policy=safe_heuristic` training ablation.** - **(E1) `cutoff_rollout_policy=heuristic_balanced` training ablation.**
Strongest single lever: gives traversal leaves stronger value estimates Strongest single lever: gives traversal leaves stronger value estimates
during training, which should ripple back to "open + follow up" signal. during training, which should ripple back to "open + follow up" signal.
Pure-self-play purity dented, but only at cutoff leaves. Pure-self-play purity dented, but only at cutoff leaves.
@@ -635,7 +635,7 @@ Fix:
Also bumped `traversal.outcome_sampling_epsilon` in `default.yaml` from Also bumped `traversal.outcome_sampling_epsilon` in `default.yaml` from
0.2 to 0.05. The 200-iteration sweep (section 1) showed 0.05 produced the 0.2 to 0.05. The 200-iteration sweep (section 1) showed 0.05 produced the
best short-run safe_heuristic_strict score diff (-40.01 vs -57.87 for best short-run heuristic_cautious score diff (-40.01 vs -57.87 for
0.20). All recent experimental runs already used 0.05; the default now 0.20). All recent experimental runs already used 0.05; the default now
matches actual experimental practice. matches actual experimental practice.
@@ -643,7 +643,71 @@ These changes do not target the diagnosed selection-bias bottleneck. They
align config intent with actual scheduler behaviour and make the default align config intent with actual scheduler behaviour and make the default
config reproduce known-best knob settings out of the box. config reproduce known-best knob settings out of the box.
### 9. Short open-selectivity ablation ### 9. Naming, plot curation, and tiered eval cadence (2026-05-10)
Hygiene changes — none target the diagnosed selection-bias bottleneck,
but they make the codebase honestly reflect the pure-self-play stance
and reduce dashboard noise.
Bot family rename (drop the unhelpful `safe_` prefix; suffixes now
describe behaviour):
| Old | New |
| --- | --- |
| `safe_heuristic_loose` | `heuristic_aggressive` |
| `safe_heuristic` | `heuristic_balanced` |
| `safe_heuristic_strict` | `heuristic_cautious` |
| `noisy_safe` | `heuristic_noisy` |
| `passive_discard` | `discard_only` |
Class renames in `bots/`: `SafeHeuristicBot``HeuristicBot`,
`SafeHeuristicParams``HeuristicParams`, `PassiveDiscardBot`
`DiscardOnlyBot`, plus the loose/strict parameter constants. Backwards
compatibility was dropped intentionally — no aliases. Active configs,
docs, scripts, tests updated; archive files (read-only by policy)
left intact and may still reference old names.
Analyze plot curation (`deep_cfr/analyze.py`):
- New `analysis_00_core.png` dashboard with 10 heuristic-free metrics
(loss/{advantage, strategy}; vs `heuristic_cautious`:
`avg_score_diff0`, `win_rate0`, `avg_opened_colors`,
`positive_expedition_rate`, `bonus_expedition_rate`,
`score_per_opened_color`, `policy_entropy`; vs `random`: `win_rate0`).
- Removed `analysis_05_open_quality.png` (bad/weak/good open rates,
recoverable score) and `analysis_07_calibration.png` (calibration gap,
recoverable mean) — both relied on the heuristic `recoverable_score`
classifier we already dropped from inputs.
- Removed `SELECTIVITY_PLOTS` and `plot_selectivity` (heuristic-laden).
- `SUMMARY_EVAL_METRICS` no longer includes `bad_open_rate` or
`calibration_gap`.
`PlotSpec` gained an optional `opponents` allowlist so the new core
section can pin a specific opponent per panel without restructuring the
existing `plot_section` plumbing.
Tiered evaluation cadence (`EvaluationConfig`):
- Added `extended_opponents: tuple[str, ...]` and `extended_eval_every:
int = 0`.
- Method `opponents_for_iteration(iteration)` returns the core list at
every `eval_every`, and appends `extended_opponents` (de-duplicated)
when `iteration` is also a multiple of `extended_eval_every`.
- `default.yaml` now uses 3 core opponents
(`random`, `discard_only`, `heuristic_cautious`) every 5 iterations
and 3 extended opponents (`heuristic_balanced`, `heuristic_aggressive`,
`heuristic_noisy`) every 50 iterations.
- `random` is the floor sanity. `discard_only` is the zero-pit
detector / absolute-score reference (its score is always 0, so
`eval/discard_only/avg_score_diff0` directly equals our model's
raw average score). `heuristic_cautious` is the ceiling and the
archive-comparable benchmark used in sections 16.
Net effect on ongoing eval cost: ~50% reduction (3 opponents × every
5 iter, plus 6 opponents × every 50 iter, vs the prior 6 opponents
× every 5).
### 10. Short open-selectivity ablation
Run a 200-300 iteration ablation only after the target audit identifies a Run a 200-300 iteration ablation only after the target audit identifies a
specific change. Candidate changes include: specific change. Candidate changes include:
@@ -656,10 +720,10 @@ specific change. Candidate changes include:
Primary metrics: Primary metrics:
- `eval/safe_heuristic_strict/avg_score_diff0` - `eval/heuristic_cautious/avg_score_diff0`
- `eval/safe_heuristic_strict/win_rate0` - `eval/heuristic_cautious/win_rate0`
- `eval/safe_heuristic_strict/bad_open_rate` - `eval/heuristic_cautious/bad_open_rate`
- `eval/safe_heuristic_strict/score_per_opened_color` - `eval/heuristic_cautious/score_per_opened_color`
Do not promote to 500+ iterations unless bad-open rate and score/opened color Do not promote to 500+ iterations unless bad-open rate and score/opened color
both improve without degrading score diff. both improve without degrading score diff.
+6 -6
View File
@@ -39,7 +39,7 @@ The experiment must produce either a recommended new `network` config or a docum
## Success criteria ## Success criteria
1. At least one tested config produces win-rate trajectories vs `safe_heuristic_strict` that are **clearly outside seed noise** compared to the current baseline at iteration 200 — OR a clear documented null result (no size in the tested range improves the curve). 1. At least one tested config produces win-rate trajectories vs `heuristic_cautious` that are **clearly outside seed noise** compared to the current baseline at iteration 200 — OR a clear documented null result (no size in the tested range improves the curve).
2. `iteration_seconds`, `traversal_seconds`, `advantage_train_seconds`, `strategy_train_seconds`, and `policy_network_seconds` (eval) are captured for each tested size and written to `docs/performance.md`. 2. `iteration_seconds`, `traversal_seconds`, `advantage_train_seconds`, `strategy_train_seconds`, and `policy_network_seconds` (eval) are captured for each tested size and written to `docs/performance.md`.
3. A recommended `network` config emerges from the data, OR the experiment documents why the current size should be kept, with specific rationale. 3. A recommended `network` config emerges from the data, OR the experiment documents why the current size should be kept, with specific rationale.
@@ -72,7 +72,7 @@ configs/deep_cfr/model-size-1536x8.yaml
- `traversal_seconds` — traversal phase. - `traversal_seconds` — traversal phase.
- `advantage_train_seconds` — advantage network optimization. - `advantage_train_seconds` — advantage network optimization.
- `strategy_train_seconds` — strategy network optimization. - `strategy_train_seconds` — strategy network optimization.
- At eval iterations {50, 100, 150, 200}: `eval/<opponent>/win_rate` for all opponents, with special attention to `safe_heuristic_strict`. - At eval iterations {50, 100, 150, 200}: `eval/<opponent>/win_rate` for all opponents, with special attention to `heuristic_cautious`.
- At eval iterations: `eval/<opponent>/policy_network_seconds` — needed for the AMP/TRT prerequisite check. - At eval iterations: `eval/<opponent>/policy_network_seconds` — needed for the AMP/TRT prerequisite check.
- **Memory monitoring:** watch GPU VRAM during the 1024x6 and 1536x8 runs. If a run OOMs or VRAM > 20 GB, reduce `optimization.advantage_batch_size` and `optimization.strategy_batch_size` by half (1024 → 512) and note the change in the results table. Do not adjust traversal settings. - **Memory monitoring:** watch GPU VRAM during the 1024x6 and 1536x8 runs. If a run OOMs or VRAM > 20 GB, reduce `optimization.advantage_batch_size` and `optimization.strategy_batch_size` by half (1024 → 512) and note the change in the results table. Do not adjust traversal settings.
@@ -259,8 +259,8 @@ if non_eval:
eval_rows = {r['iteration']: r for r in rows if r.get('evaluation_seconds')} eval_rows = {r['iteration']: r for r in rows if r.get('evaluation_seconds')}
for it in [50, 100, 150, 200]: for it in [50, 100, 150, 200]:
if it in eval_rows: if it in eval_rows:
wr = eval_rows[it].get('eval/safe_heuristic_strict/win_rate', 'n/a') wr = eval_rows[it].get('eval/heuristic_cautious/win_rate', 'n/a')
print(f' iter={it} safe_heuristic_strict win_rate={wr}') print(f' iter={it} heuristic_cautious win_rate={wr}')
" "
done done
``` ```
@@ -269,7 +269,7 @@ done
After the grid completes, append a date-stamped experiment subsection to `docs/performance.md` under the "Experiments" heading. The subsection must include: After the grid completes, append a date-stamped experiment subsection to `docs/performance.md` under the "Experiments" heading. The subsection must include:
- A results table with `iteration_seconds` mean (non-eval) and win-rate vs `safe_heuristic_strict` at {50, 100, 150, 200} for each config. - A results table with `iteration_seconds` mean (non-eval) and win-rate vs `heuristic_cautious` at {50, 100, 150, 200} for each config.
- A `policy_network_seconds` column from eval rows — this is the key data for the AMP/compile/TRT prerequisite check. - A `policy_network_seconds` column from eval rows — this is the key data for the AMP/compile/TRT prerequisite check.
- The recommendation that follows from the decision tree below. - The recommendation that follows from the decision tree below.
@@ -302,7 +302,7 @@ Apply this logic after the grid completes:
### Branch A — a size unlocks the curve AND iter time is acceptable ### Branch A — a size unlocks the curve AND iter time is acceptable
**Condition:** at least one config at or above 768x4 shows win-rate trajectories vs `safe_heuristic_strict` that are clearly outside seed noise vs 512x3 baseline at iteration 200, AND `iteration_seconds` at that size is ≤ 3× the baseline (i.e., ≤ ~54s/iter). **Condition:** at least one config at or above 768x4 shows win-rate trajectories vs `heuristic_cautious` that are clearly outside seed noise vs 512x3 baseline at iteration 200, AND `iteration_seconds` at that size is ≤ 3× the baseline (i.e., ≤ ~54s/iter).
**Action:** **Action:**
1. Recommend that config as the new `network` default. 1. Recommend that config as the new `network` default.
+2 -2
View File
@@ -152,7 +152,7 @@ Cython production rewrite. It intentionally uses per-context RNG so interleaved
execution order does not change the random stream for another context. That execution order does not change the random stream for another context. That
lets the prototype assert value/stat/sample parity against a recursive prototype lets the prototype assert value/stat/sample parity against a recursive prototype
while measuring realized batch size. Production Cython parity is a later Phase 2 while measuring realized batch size. Production Cython parity is a later Phase 2
gate because the real path also has safe-heuristic opponents, average-strategy gate because the real path also has heuristic-balanced opponents, average-strategy
opponents, self-play league snapshots, deck-draw chance sampling, external opponents, self-play league snapshots, deck-draw chance sampling, external
sampling, and cutoff rollouts. sampling, and cutoff rollouts.
@@ -312,7 +312,7 @@ Required feature expansion:
- Support `opponent_policy: average_strategy`, matching the default config's - Support `opponent_policy: average_strategy`, matching the default config's
opponent branch. opponent branch.
- Keep unsupported branches guarded (`self_play_league`, `safe_heuristic`, - Keep unsupported branches guarded (`self_play_league`, `heuristic_balanced`,
random rollout cutoffs, external sampling). random rollout cutoffs, external sampling).
- Add parity tests for the average-strategy fixed-opponent branch. - Add parity tests for the average-strategy fixed-opponent branch.
- Verify a default-policy interleaved run starts and emits batch metrics. - Verify a default-policy interleaved run starts and emits batch metrics.
+1 -1
View File
@@ -26,7 +26,7 @@ Re-enable `torch.compile` on the Deep CFR trainer's networks at a model size whe
1. **Model-size precondition met.** The active `default.yaml` (or the targeted variant) has `network.hidden_size ≥ 1024` *or* `network.num_layers ≥ 6`, *or* an architecture (e.g. `color_shared` with non-trivial `color_attention_layers`) whose per-call forward time exceeds ~150 μs at the trainer's training batch size on the target GPU. If neither condition holds, this plan is **not merged**; the branch is parked. 1. **Model-size precondition met.** The active `default.yaml` (or the targeted variant) has `network.hidden_size ≥ 1024` *or* `network.num_layers ≥ 6`, *or* an architecture (e.g. `color_shared` with non-trivial `color_attention_layers`) whose per-call forward time exceeds ~150 μs at the trainer's training batch size on the target GPU. If neither condition holds, this plan is **not merged**; the branch is parked.
2. **Iter-time improvement.** With `compile.trainer.enabled: true` on the chosen larger model and the same seed, the 1000-iter projection improves by at least **5%** vs the no-compile baseline on the same machine (measured on `home`). Eval and checkpointing should be disabled for the bench window, matching the protocol used in the 2026-05-07 experiment. 2. **Iter-time improvement.** With `compile.trainer.enabled: true` on the chosen larger model and the same seed, the 1000-iter projection improves by at least **5%** vs the no-compile baseline on the same machine (measured on `home`). Eval and checkpointing should be disabled for the bench window, matching the protocol used in the 2026-05-07 experiment.
3. **No learning-curve drift.** Over at least 100 iterations with `compile.trainer.enabled: true` vs `false` (same seed, same config), the eval win-rate trajectories against `random` and `safe_heuristic` are within seed noise. If trajectories visibly diverge, the plan does not ship even if iter time improves. 3. **No learning-curve drift.** Over at least 100 iterations with `compile.trainer.enabled: true` vs `false` (same seed, same config), the eval win-rate trajectories against `random` and `heuristic_balanced` are within seed noise. If trajectories visibly diverge, the plan does not ship even if iter time improves.
4. **No checkpoint-format break.** Checkpoints saved with compile enabled must load cleanly when compile is disabled, and vice versa. (Handled via `_clean_state_dict()`; see Risks.) 4. **No checkpoint-format break.** Checkpoints saved with compile enabled must load cleanly when compile is disabled, and vice versa. (Handled via `_clean_state_dict()`; see Risks.)
5. **No multiprocessing-worker break.** Whether `inference_backend` is `local` or `server`, traversal workers must continue to receive uncompiled `state_dict`s without `_orig_mod.` prefixes. 5. **No multiprocessing-worker break.** Whether `inference_backend` is `local` or `server`, traversal workers must continue to receive uncompiled `state_dict`s without `_orig_mod.` prefixes.
6. **(Secondary) Inference-server forward.** If step B below is taken, the server's `policy_network_seconds` decreases by at least 20% at the chosen model size, with no traversal-path correctness regression. If step B does not produce a measurable win, it is left disabled and the plan still ships with step A only. 6. **(Secondary) Inference-server forward.** If step B below is taken, the server's `policy_network_seconds` decreases by at least 20% at the chosen model size, with no traversal-path correctness regression. If step B does not produce a measurable win, it is left disabled and the plan still ships with step A only.
@@ -157,9 +157,9 @@ Python-object touch가 깔려 있다 (per-node, per-iteration):
`self.strategy_samples.append(...)` (file:903, 937, 969). list의 `self.strategy_samples.append(...)` (file:903, 937, 969). list의
PyObject reference 갱신은 free-threaded Python에서도 atomic refcount PyObject reference 갱신은 free-threaded Python에서도 atomic refcount
비용을 추가로 부담한다. 비용을 추가로 부담한다.
6. **`SafeHeuristicBot.act(state)`** — `_fixed_opponent_action` 6. **`HeuristicBot.act(state)`** — `_fixed_opponent_action`
(file:633, 652), `_rollout_value` (file:841). Python class (file:633, 652), `_rollout_value` (file:841). Python class
메서드 호출. `safe_heuristic` 옵션 사용 시만 핫. 메서드 호출. `heuristic_balanced` 옵션 사용 시만 핫.
7. **`league_advantage_networks` indexing** — `_self_play_snapshot_ 7. **`league_advantage_networks` indexing** — `_self_play_snapshot_
networks` (file:802), `[-recent_count:]`, `[:max(0, ...)]` slicing networks` (file:802), `[-recent_count:]`, `[:max(0, ...)]` slicing
= Python list slicing. = Python list slicing.
@@ -267,15 +267,15 @@ candidates = self.league_advantage_networks[:max(0, len(self.league_advantage_ne
- 빈도: traversal 진입 시 한 번 (`traverse`에서 미리 픽), 재귀 안에서는 - 빈도: traversal 진입 시 한 번 (`traverse`에서 미리 픽), 재귀 안에서는
`active_self_play_networks`만 본다. 따라서 cold path. 변환 불필요. `active_self_play_networks`만 본다. 따라서 cold path. 변환 불필요.
### B6. `SafeHeuristicBot.act(state)` — Python bot ### B6. `HeuristicBot.act(state)` — Python bot
```python ```python
# traversal.pyx:633, 652, 841 # traversal.pyx:633, 652, 841
return int(self.safe_heuristic_opponent_bot.act(state)) return int(self.heuristic_opponent_bot.act(state))
``` ```
- 빈도: `opponent_policy=safe_heuristic` 또는 `cutoff_rollout_policy= - 빈도: `opponent_policy=heuristic_balanced` 또는 `cutoff_rollout_policy=
safe_heuristic`일 때만. 현 default는 self_play_league + score_diff heuristic_balanced`일 때만. 현 default는 self_play_league + score_diff
cutoff (per memory의 opponent_policy_network_divergence note + AGENTS). cutoff (per memory의 opponent_policy_network_divergence note + AGENTS).
- 변환 난이도: **Medium-High** (Python class 전체를 cython화). 현 default - 변환 난이도: **Medium-High** (Python class 전체를 cython화). 현 default
config에서는 핫 아님 — 시도하지 않는 게 합리. config에서는 핫 아님 — 시도하지 않는 게 합리.
+1 -1
View File
@@ -48,7 +48,7 @@ src/coolrl_lost_cities/games/classic/deep_cfr/
traversal_stats.py — structured traversal diagnostic metrics traversal_stats.py — structured traversal diagnostic metrics
# Auxiliary training modes # Auxiliary training modes
imitation.py — safe-heuristic imitation pretraining imitation.py — heuristic imitation pretraining
policy_gradient.py — policy-gradient fine-tuning policy_gradient.py — policy-gradient fine-tuning
``` ```
@@ -59,23 +59,23 @@ Evaluation performance peaks early and then degrades.
| Opponent | First 20 evals `win_rate0` | Last 20 evals `win_rate0` | First 20 evals score diff | Last 20 evals score diff | | Opponent | First 20 evals `win_rate0` | Last 20 evals `win_rate0` | First 20 evals score diff | Last 20 evals score diff |
| --- | ---: | ---: | ---: | ---: | | --- | ---: | ---: | ---: | ---: |
| random | 0.808 | 0.665 | +35.1 | +12.4 | | random | 0.808 | 0.665 | +35.1 | +12.4 |
| passive_discard | 0.026 | 0.012 | -30.8 | -43.0 | | discard_only | 0.026 | 0.012 | -30.8 | -43.0 |
| safe_heuristic | 0.070 | 0.011 | -68.7 | -94.2 | | heuristic_balanced | 0.070 | 0.011 | -68.7 | -94.2 |
| safe_heuristic_loose | 0.078 | 0.014 | -69.3 | -95.0 | | heuristic_aggressive | 0.078 | 0.014 | -69.3 | -95.0 |
| safe_heuristic_strict | 0.072 | 0.011 | -57.6 | -85.0 | | heuristic_cautious | 0.072 | 0.011 | -57.6 | -85.0 |
| noisy_safe | 0.120 | 0.033 | -49.4 | -78.8 | | heuristic_noisy | 0.120 | 0.033 | -49.4 | -78.8 |
The best strict-heuristic point appears around iteration 70: The best strict-heuristic point appears around iteration 70:
- `eval/safe_heuristic_strict/win_rate0=0.15` - `eval/heuristic_cautious/win_rate0=0.15`
- `eval/safe_heuristic_strict/avg_score_diff0=-41.09` - `eval/heuristic_cautious/avg_score_diff0=-41.09`
The final point at iteration 2000 is worse: The final point at iteration 2000 is worse:
- `eval/safe_heuristic_strict/win_rate0=0.02` - `eval/heuristic_cautious/win_rate0=0.02`
- `eval/safe_heuristic_strict/avg_score_diff0=-81.64` - `eval/heuristic_cautious/avg_score_diff0=-81.64`
Selectivity does not meaningfully emerge. Against `safe_heuristic_strict`: Selectivity does not meaningfully emerge. Against `heuristic_cautious`:
| Metric | First 20 evals | Last 20 evals | Interpretation | | Metric | First 20 evals | Last 20 evals | Interpretation |
| --- | ---: | ---: | --- | | --- | ---: | ---: | --- |
+1 -1
View File
@@ -19,7 +19,7 @@ The primary bottleneck in CUDA-based evaluation is the overhead of launching sma
A critical refinement in the batched implementation was the handling of policy entropy. Initial versions that calculated entropy per-row on the CPU incurred significant synchronization penalties because each row required a GPU-to-CPU transfer. Moving the entropy calculation into the Torch post-processing pipeline—specifically calculating it directly on the `probs_tensor` (line 220)—ensures that the computation remains on the device and only the final results are transferred back to the host in bulk. A critical refinement in the batched implementation was the handling of policy entropy. Initial versions that calculated entropy per-row on the CPU incurred significant synchronization penalties because each row required a GPU-to-CPU transfer. Moving the entropy calculation into the Torch post-processing pipeline—specifically calculating it directly on the `probs_tensor` (line 220)—ensures that the computation remains on the device and only the final results are transferred back to the host in bulk.
Once network inference is batched, the remaining bottleneck often shifts to the CPU-bound logic of heuristic opponents (e.g., `safe_heuristic_strict`). Parallelizing the evaluation across multiple workers allows the trainer to evaluate against multiple opponents simultaneously. For a single iteration profile, using 4 parallel workers reduced wall-clock evaluation time from 14.8 seconds to 6.4 seconds, achieving a ~2.3x speedup. Once network inference is batched, the remaining bottleneck often shifts to the CPU-bound logic of heuristic opponents (e.g., `heuristic_cautious`). Parallelizing the evaluation across multiple workers allows the trainer to evaluate against multiple opponents simultaneously. For a single iteration profile, using 4 parallel workers reduced wall-clock evaluation time from 14.8 seconds to 6.4 seconds, achieving a ~2.3x speedup.
## Practical implication ## Practical implication
+1 -1
View File
@@ -37,7 +37,7 @@ On CPU, the `network / turn` cost is approximately **0.074 ms**. On CUDA, this r
### Secondary Bottlenecks ### Secondary Bottlenecks
- **Post-processing:** Moving tensors back to CPU (`.cpu().numpy()`) and calculating entropy adds measurable overhead on CUDA that is largely absent on CPU. - **Post-processing:** Moving tensors back to CPU (`.cpu().numpy()`) and calculating entropy adds measurable overhead on CUDA that is largely absent on CPU.
- **Opponent Logic:** Heuristic opponents (e.g., `safe_heuristic`) contribute significant `opponent_act_seconds` (up to 3.5s per eval iteration). Since this logic is pure Python/Cython and runs on the CPU, it does not benefit from GPU acceleration, further diluting any potential CUDA wins. - **Opponent Logic:** Heuristic opponents (e.g., `heuristic_balanced`) contribute significant `opponent_act_seconds` (up to 3.5s per eval iteration). Since this logic is pure Python/Cython and runs on the CPU, it does not benefit from GPU acceleration, further diluting any potential CUDA wins.
## Practical Implications ## Practical Implications
+2 -2
View File
@@ -45,7 +45,7 @@ Two temporary runs used:
- `run.max_iterations=3` - `run.max_iterations=3`
- `evaluation.eval_every=1` - `evaluation.eval_every=1`
- `evaluation.games=20` - `evaluation.games=20`
- `evaluation.opponents=[random,safe_heuristic_strict]` - `evaluation.opponents=[random,heuristic_cautious]`
- W&B disabled - W&B disabled
Runs: Runs:
@@ -55,7 +55,7 @@ Runs:
Core metrics matched exactly: Core metrics matched exactly:
| Iteration | `traversal/nodes` | `memory/advantage` | `loss/advantage` | `eval/random/win_rate0` | `eval/safe_heuristic_strict/win_rate0` | | Iteration | `traversal/nodes` | `memory/advantage` | `loss/advantage` | `eval/random/win_rate0` | `eval/heuristic_cautious/win_rate0` |
| --- | ---: | ---: | ---: | ---: | ---: | | --- | ---: | ---: | ---: | ---: | ---: |
| 1 | 170822 | 85096 | 803.9475702643394 | 0.75 | 0.10 | | 1 | 170822 | 85096 | 803.9475702643394 | 0.75 | 0.10 |
| 2 | 206225 | 187987 | 821.3355012834072 | 0.50 | 0.05 | | 2 | 206225 | 187987 | 821.3355012834072 | 0.50 | 0.05 |
+1 -1
View File
@@ -56,7 +56,7 @@ registered classic bots (`evaluate.py`), training CLI and evaluation CLI
(`cli.py`), traversal benchmark CLI (`benchmark.py`), `metrics.jsonl` / (`cli.py`), traversal benchmark CLI (`benchmark.py`), `metrics.jsonl` /
`runtime_progress.json` / `train.log` run artifacts, self-play league with `runtime_progress.json` / `train.log` run artifacts, self-play league with
snapshot pool and weighted current/recent/older/anchor bucket sampling, safe- snapshot pool and weighted current/recent/older/anchor bucket sampling, safe-
heuristic anchor opponent, safe-heuristic imitation pretraining (`imitation.py`), heuristic anchor opponent, heuristic imitation pretraining (`imitation.py`),
and policy-gradient fine-tuning (`policy_gradient.py`). and policy-gradient fine-tuning (`policy_gradient.py`).
As of `ad0be89`, the package also includes `inference_server.py`, As of `ad0be89`, the package also includes `inference_server.py`,
+1 -1
View File
@@ -31,7 +31,7 @@ Decision criteria for going forward:
### 2026-05-07 — Safe heuristic single-thread parity (criterion 1) ### 2026-05-07 — Safe heuristic single-thread parity (criterion 1)
Path: `experiments/julia_safe_heuristic/`. Path: `experiments/julia_heuristic/`.
1,838 snapshots in 157.471 ms median (~85.6 μs/call). Action-sequence 1,838 snapshots in 157.471 ms median (~85.6 μs/call). Action-sequence
parity vs Python. Same order of magnitude as the Cython port of the parity vs Python. Same order of magnitude as the Cython port of the
+1 -1
View File
@@ -28,7 +28,7 @@ Deep CFR은 최종적으로 average strategy가 수렴 대상. current advantage
Visible-score predictor의 ceiling은 있을 수 있음. 다만 selectivity 자체의 ceiling은 아님. selectivity는 visible-score prediction 말고 option value, irreversible cost 회피, opponent dynamics 대응 등 다른 경로로도 emerge 가능. Visible-score predictor의 ceiling은 있을 수 있음. 다만 selectivity 자체의 ceiling은 아님. selectivity는 visible-score prediction 말고 option value, irreversible cost 회피, opponent dynamics 대응 등 다른 경로로도 emerge 가능.
**Self-play attractor 가설** **Self-play attractor 가설**
5-color가 stable equilibrium. 한쪽이 selectivity 시도하면 즉시 손해. safe_heuristic이 3.7에서 강하면 진짜 NE는 아닐 듯. 다만 self-play 안에서는 stable. 5-color가 stable equilibrium. 한쪽이 selectivity 시도하면 즉시 손해. heuristic_balanced이 3.7에서 강하면 진짜 NE는 아닐 듯. 다만 self-play 안에서는 stable.
**Lost Cities NE 자체가 5-color 가설** **Lost Cities NE 자체가 5-color 가설**
이론적 가능성 0 아님. self-play가 발견한 게 진짜 NE면 transition 영원히 안 옴. 검증 가능한 형태로는 tabular oracle 또는 BR 진단. 이론적 가능성 0 아님. self-play가 발견한 게 진짜 NE면 transition 영원히 안 옴. 검증 가능한 형태로는 tabular oracle 또는 BR 진단.
+1 -1
View File
@@ -240,7 +240,7 @@ def main() -> None:
parser.add_argument("--output", type=Path, required=True) parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args() args = parser.parse_args()
opponents = args.opponent or ["safe_heuristic_strict"] opponents = args.opponent or ["heuristic_cautious"]
args.output.parent.mkdir(parents=True, exist_ok=True) args.output.parent.mkdir(parents=True, exist_ok=True)
rows = [] rows = []
for checkpoint in args.checkpoints: for checkpoint in args.checkpoints:
+2 -2
View File
@@ -367,7 +367,7 @@ def analyze_checkpoint(
def main() -> None: def main() -> None:
parser = argparse.ArgumentParser(description=__doc__) parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("checkpoints", nargs="+", type=Path) parser.add_argument("checkpoints", nargs="+", type=Path)
parser.add_argument("--opponent", default="safe_heuristic_strict") parser.add_argument("--opponent", default="heuristic_cautious")
parser.add_argument("--games", type=int, default=100) parser.add_argument("--games", type=int, default=100)
parser.add_argument("--seed", type=int, default=231_000) parser.add_argument("--seed", type=int, default=231_000)
parser.add_argument("--device", default="cuda") parser.add_argument("--device", default="cuda")
@@ -379,7 +379,7 @@ def main() -> None:
help=( help=(
"Policy used for the policy_player during forced-action rollouts. " "Policy used for the policy_player during forced-action rollouts. "
"'model' uses the trained advantage network; any other value is " "'model' uses the trained advantage network; any other value is "
"treated as a bot name (e.g. 'safe_heuristic_strict')." "treated as a bot name (e.g. 'heuristic_cautious')."
), ),
) )
parser.add_argument("--output", type=Path, required=True) parser.add_argument("--output", type=Path, required=True)
+1 -1
View File
@@ -273,7 +273,7 @@ def analyze_checkpoint(
def main() -> None: def main() -> None:
parser = argparse.ArgumentParser(description=__doc__) parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("checkpoints", nargs="+", type=Path) parser.add_argument("checkpoints", nargs="+", type=Path)
parser.add_argument("--opponent", default="safe_heuristic_strict") parser.add_argument("--opponent", default="heuristic_cautious")
parser.add_argument("--games", type=int, default=100) parser.add_argument("--games", type=int, default=100)
parser.add_argument("--seed", type=int, default=232_000) parser.add_argument("--seed", type=int, default=232_000)
parser.add_argument("--device", default="cpu") parser.add_argument("--device", default="cpu")
+1 -1
View File
@@ -265,7 +265,7 @@ def main() -> None:
parser.add_argument("--output", type=Path, required=True) parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args() args = parser.parse_args()
opponents = args.opponent or ["random", "safe_heuristic_strict"] opponents = args.opponent or ["random", "heuristic_cautious"]
args.output.parent.mkdir(parents=True, exist_ok=True) args.output.parent.mkdir(parents=True, exist_ok=True)
rows: list[dict] = [] rows: list[dict] = []
+7 -7
View File
@@ -7,22 +7,22 @@ from typing import Any
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig
from coolrl_lost_cities.games.classic.bots.heuristic_py import SafeHeuristicBot from coolrl_lost_cities.games.classic.bots.heuristic_py import HeuristicBot
from coolrl_lost_cities.games.classic.bots.registry import ( from coolrl_lost_cities.games.classic.bots.registry import (
LOOSE_SAFE_HEURISTIC_PARAMS, AGGRESSIVE_HEURISTIC_PARAMS,
STRICT_SAFE_HEURISTIC_PARAMS, CAUTIOUS_HEURISTIC_PARAMS,
) )
VARIANTS = { VARIANTS = {
"default": None, "default": None,
"loose": LOOSE_SAFE_HEURISTIC_PARAMS, "loose": AGGRESSIVE_HEURISTIC_PARAMS,
"strict": STRICT_SAFE_HEURISTIC_PARAMS, "strict": CAUTIOUS_HEURISTIC_PARAMS,
} }
def parse_args() -> argparse.Namespace: def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description="Export safe-heuristic bot parity snapshots for external implementations." description="Export heuristic bot parity snapshots for external implementations."
) )
parser.add_argument("--output", required=True, help="JSONL output path.") parser.add_argument("--output", required=True, help="JSONL output path.")
parser.add_argument("--seeds", type=int, default=50, help="Number of seeds per config.") parser.add_argument("--seeds", type=int, default=50, help="Number of seeds per config.")
@@ -71,7 +71,7 @@ def main() -> None:
for config_name, config in _configs(): for config_name, config in _configs():
for variant_name, params in VARIANTS.items(): for variant_name, params in VARIANTS.items():
for seed in range(args.seeds): for seed in range(args.seeds):
bot = SafeHeuristicBot(params) bot = HeuristicBot(params)
state = GameState.new_game(config, seed=seed) state = GameState.new_game(config, seed=seed)
for turn in range(args.max_steps): for turn in range(args.max_steps):
if state.terminal: if state.terminal:
@@ -1,8 +1,8 @@
from __future__ import annotations from __future__ import annotations
from ..policy import LostCitiesPolicy, PolicyInput from ..policy import LostCitiesPolicy, PolicyInput
from .heuristic import SafeHeuristicBot from .discard_only import DiscardOnlyBot
from .passive import PassiveDiscardBot from .heuristic import HeuristicBot
from .random import RandomBot from .random import RandomBot
from .registry import DEFAULT_BOT, available_bot_names, build_bot from .registry import DEFAULT_BOT, available_bot_names, build_bot
@@ -10,9 +10,9 @@ __all__ = [
"PolicyInput", "PolicyInput",
"DEFAULT_BOT", "DEFAULT_BOT",
"LostCitiesPolicy", "LostCitiesPolicy",
"PassiveDiscardBot", "DiscardOnlyBot",
"RandomBot", "RandomBot",
"SafeHeuristicBot", "HeuristicBot",
"available_bot_names", "available_bot_names",
"build_bot", "build_bot",
] ]
@@ -6,7 +6,7 @@ from ..snapshots import Snapshot
from .base import first_legal, legal_from_obs from .base import first_legal, legal_from_obs
class PassiveDiscardBot(LostCitiesPolicy): class DiscardOnlyBot(LostCitiesPolicy):
"""Baseline that avoids opening expeditions whenever discarding is legal.""" """Baseline that avoids opening expeditions whenever discarding is legal."""
def act(self, obs_or_state: PolicyInput) -> int: def act(self, obs_or_state: PolicyInput) -> int:
@@ -4,8 +4,8 @@ from .heuristic_cy import (
DRAW_FROM_DECK_ACTION, DRAW_FROM_DECK_ACTION,
PLAY_OR_DISCARD_ACTIONS_PER_SLOT, PLAY_OR_DISCARD_ACTIONS_PER_SLOT,
DerivedHeuristicConfig, DerivedHeuristicConfig,
SafeHeuristicBot, HeuristicBot,
SafeHeuristicParams, HeuristicParams,
derive_heuristic_config, derive_heuristic_config,
discard_action, discard_action,
draw_from_discard_action, draw_from_discard_action,
@@ -16,8 +16,8 @@ __all__ = [
"DRAW_FROM_DECK_ACTION", "DRAW_FROM_DECK_ACTION",
"PLAY_OR_DISCARD_ACTIONS_PER_SLOT", "PLAY_OR_DISCARD_ACTIONS_PER_SLOT",
"DerivedHeuristicConfig", "DerivedHeuristicConfig",
"SafeHeuristicBot", "HeuristicBot",
"SafeHeuristicParams", "HeuristicParams",
"derive_heuristic_config", "derive_heuristic_config",
"discard_action", "discard_action",
"draw_from_discard_action", "draw_from_discard_action",
@@ -31,7 +31,7 @@ def draw_from_discard_action(color: int) -> int:
return 1 + color return 1 + color
LOGGER = logging.getLogger("coolrl_lost_cities.games.classic.bots.safe_heuristic") LOGGER = logging.getLogger("coolrl_lost_cities.games.classic.bots.heuristic")
class _CachedState: class _CachedState:
@@ -66,7 +66,7 @@ class _CachedState:
@dataclass(frozen=True) @dataclass(frozen=True)
class SafeHeuristicParams: class HeuristicParams:
# Expedition opening. # Expedition opening.
open_target_ratio: float = 0.50 open_target_ratio: float = 0.50
open_min_card_ratio: float = 0.40 open_min_card_ratio: float = 0.40
@@ -124,7 +124,7 @@ class DerivedHeuristicConfig:
@lru_cache(maxsize=64) @lru_cache(maxsize=64)
def derive_heuristic_config( def derive_heuristic_config(
config: LostCitiesConfig, config: LostCitiesConfig,
params: SafeHeuristicParams, params: HeuristicParams,
) -> DerivedHeuristicConfig: ) -> DerivedHeuristicConfig:
max_color_sum = sum(config.min_rank + rank - 1 for rank in range(1, config.n_ranks + 1)) max_color_sum = sum(config.min_rank + rank - 1 for rank in range(1, config.n_ranks + 1))
break_even_sum = -config.expedition_penalty break_even_sum = -config.expedition_penalty
@@ -178,20 +178,20 @@ def derive_heuristic_config(
) )
class SafeHeuristicBot(LostCitiesPolicy): class HeuristicBot(LostCitiesPolicy):
def __init__(self, params: SafeHeuristicParams | None = None): def __init__(self, params: HeuristicParams | None = None):
self.params = params or SafeHeuristicParams() self.params = params or HeuristicParams()
def act(self, obs_or_state: PolicyInput) -> int: def act(self, obs_or_state: PolicyInput) -> int:
if not isinstance(obs_or_state, GameState) and not hasattr(obs_or_state, "legal_mask"): if not isinstance(obs_or_state, GameState) and not hasattr(obs_or_state, "legal_mask"):
LOGGER.debug( LOGGER.debug(
"SafeHeuristicBot fallback to first legal: input_type=%s", "HeuristicBot fallback to first legal: input_type=%s",
type(obs_or_state).__name__, type(obs_or_state).__name__,
) )
return first_legal(legal_from_obs(obs_or_state)) return first_legal(legal_from_obs(obs_or_state))
LOGGER.debug( LOGGER.debug(
"SafeHeuristicBot heuristic path: player=%s phase=%s turn=%s", "HeuristicBot heuristic path: player=%s phase=%s turn=%s",
obs_or_state.current_player, obs_or_state.current_player,
obs_or_state.phase, obs_or_state.phase,
obs_or_state.turn_count, obs_or_state.turn_count,
@@ -30,11 +30,11 @@ def draw_from_discard_action(color: int) -> int:
return 1 + color return 1 + color
LOGGER = logging.getLogger("coolrl_lost_cities.games.classic.bots.safe_heuristic") LOGGER = logging.getLogger("coolrl_lost_cities.games.classic.bots.heuristic")
@dataclass(frozen=True) @dataclass(frozen=True)
class SafeHeuristicParams: class HeuristicParams:
# Expedition opening. # Expedition opening.
open_target_ratio: float = 0.50 open_target_ratio: float = 0.50
open_min_card_ratio: float = 0.40 open_min_card_ratio: float = 0.40
@@ -92,7 +92,7 @@ class DerivedHeuristicConfig:
@lru_cache(maxsize=64) @lru_cache(maxsize=64)
def derive_heuristic_config( def derive_heuristic_config(
config: LostCitiesConfig, config: LostCitiesConfig,
params: SafeHeuristicParams, params: HeuristicParams,
) -> DerivedHeuristicConfig: ) -> DerivedHeuristicConfig:
max_color_sum = sum(config.min_rank + rank - 1 for rank in range(1, config.n_ranks + 1)) max_color_sum = sum(config.min_rank + rank - 1 for rank in range(1, config.n_ranks + 1))
break_even_sum = -config.expedition_penalty break_even_sum = -config.expedition_penalty
@@ -146,20 +146,20 @@ def derive_heuristic_config(
) )
class SafeHeuristicBot(LostCitiesPolicy): class HeuristicBot(LostCitiesPolicy):
def __init__(self, params: SafeHeuristicParams | None = None): def __init__(self, params: HeuristicParams | None = None):
self.params = params or SafeHeuristicParams() self.params = params or HeuristicParams()
def act(self, obs_or_state: PolicyInput) -> int: def act(self, obs_or_state: PolicyInput) -> int:
if not isinstance(obs_or_state, GameState) and not hasattr(obs_or_state, "legal_mask"): if not isinstance(obs_or_state, GameState) and not hasattr(obs_or_state, "legal_mask"):
LOGGER.debug( LOGGER.debug(
"SafeHeuristicBot fallback to first legal: input_type=%s", "HeuristicBot fallback to first legal: input_type=%s",
type(obs_or_state).__name__, type(obs_or_state).__name__,
) )
return first_legal(legal_from_obs(obs_or_state)) return first_legal(legal_from_obs(obs_or_state))
LOGGER.debug( LOGGER.debug(
"SafeHeuristicBot heuristic path: player=%s phase=%s turn=%s", "HeuristicBot heuristic path: player=%s phase=%s turn=%s",
obs_or_state.current_player, obs_or_state.current_player,
obs_or_state.phase, obs_or_state.phase,
obs_or_state.turn_count, obs_or_state.turn_count,
@@ -3,8 +3,8 @@ from __future__ import annotations
from collections.abc import Callable from collections.abc import Callable
from ..policy import LostCitiesPolicy, PolicyInput from ..policy import LostCitiesPolicy, PolicyInput
from .heuristic import SafeHeuristicBot, SafeHeuristicParams from .discard_only import DiscardOnlyBot
from .passive import PassiveDiscardBot from .heuristic import HeuristicBot, HeuristicParams
from .random import RandomBot from .random import RandomBot
BotName = str BotName = str
@@ -30,7 +30,7 @@ class NoisyPolicy(LostCitiesPolicy):
return self.base.act(obs_or_state) return self.base.act(obs_or_state)
LOOSE_SAFE_HEURISTIC_PARAMS = SafeHeuristicParams( AGGRESSIVE_HEURISTIC_PARAMS = HeuristicParams(
open_target_ratio=0.42, open_target_ratio=0.42,
open_min_card_ratio=0.30, open_min_card_ratio=0.30,
handshake_target_multiplier=1.00, handshake_target_multiplier=1.00,
@@ -38,7 +38,7 @@ LOOSE_SAFE_HEURISTIC_PARAMS = SafeHeuristicParams(
late_open_block_ratio=0.12, late_open_block_ratio=0.12,
) )
STRICT_SAFE_HEURISTIC_PARAMS = SafeHeuristicParams( CAUTIOUS_HEURISTIC_PARAMS = HeuristicParams(
open_target_ratio=0.62, open_target_ratio=0.62,
open_min_card_ratio=0.50, open_min_card_ratio=0.50,
handshake_target_multiplier=1.35, handshake_target_multiplier=1.35,
@@ -49,12 +49,12 @@ STRICT_SAFE_HEURISTIC_PARAMS = SafeHeuristicParams(
BOT_REGISTRY: dict[BotName, PolicyFactory] = { BOT_REGISTRY: dict[BotName, PolicyFactory] = {
DEFAULT_BOT: RandomBot, DEFAULT_BOT: RandomBot,
"passive-discard": lambda seed: PassiveDiscardBot(), "discard-only": lambda seed: DiscardOnlyBot(),
"safe-heuristic": lambda seed: SafeHeuristicBot(), "heuristic-balanced": lambda seed: HeuristicBot(),
"safe-heuristic-loose": lambda seed: SafeHeuristicBot(LOOSE_SAFE_HEURISTIC_PARAMS), "heuristic-aggressive": lambda seed: HeuristicBot(AGGRESSIVE_HEURISTIC_PARAMS),
"safe-heuristic-strict": lambda seed: SafeHeuristicBot(STRICT_SAFE_HEURISTIC_PARAMS), "heuristic-cautious": lambda seed: HeuristicBot(CAUTIOUS_HEURISTIC_PARAMS),
"noisy-safe": lambda seed: NoisyPolicy( "heuristic-noisy": lambda seed: NoisyPolicy(
SafeHeuristicBot(), HeuristicBot(),
RandomBot(seed), RandomBot(seed),
), ),
} }
@@ -18,6 +18,7 @@ class PlotSpec:
scale: float = 1.0 scale: float = 1.0
kind: str = "eval" kind: str = "eval"
fixed_ylim: tuple[float, float] | None = None fixed_ylim: tuple[float, float] | None = None
opponents: tuple[str, ...] | None = None
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -28,6 +29,71 @@ class SectionSpec:
SECTIONS: tuple[SectionSpec, ...] = ( SECTIONS: tuple[SectionSpec, ...] = (
SectionSpec(
"Core",
"analysis_00_core.png",
(
PlotSpec("Advantage Loss", ("loss/advantage",), "loss", kind="train"),
PlotSpec("Strategy Loss", ("loss/strategy",), "loss", kind="train"),
PlotSpec(
"Avg Score Diff (heuristic_cautious)",
("avg_score_diff0",),
"score diff",
opponents=("heuristic_cautious",),
),
PlotSpec(
"Win Rate (heuristic_cautious)",
("win_rate0",),
"rate (%)",
scale=100.0,
fixed_ylim=(0, 100),
opponents=("heuristic_cautious",),
),
PlotSpec(
"Win Rate (random)",
("win_rate0",),
"rate (%)",
scale=100.0,
fixed_ylim=(0, 100),
opponents=("random",),
),
PlotSpec(
"Avg Opened Colors (heuristic_cautious)",
("avg_opened_colors",),
"colors",
fixed_ylim=(0, 5),
opponents=("heuristic_cautious",),
),
PlotSpec(
"Positive Expedition Rate (heuristic_cautious)",
("positive_expedition_rate",),
"rate (%)",
scale=100.0,
fixed_ylim=(0, 100),
opponents=("heuristic_cautious",),
),
PlotSpec(
"Bonus Expedition Rate (heuristic_cautious)",
("bonus_expedition_rate",),
"rate (%)",
scale=100.0,
fixed_ylim=(0, 100),
opponents=("heuristic_cautious",),
),
PlotSpec(
"Score per Opened Color (heuristic_cautious)",
("score_per_opened_color",),
"score / color",
opponents=("heuristic_cautious",),
),
PlotSpec(
"Policy Entropy (heuristic_cautious)",
("policy_entropy",),
"entropy",
opponents=("heuristic_cautious",),
),
),
),
SectionSpec( SectionSpec(
"Loss", "Loss",
"analysis_01_loss.png", "analysis_01_loss.png",
@@ -107,48 +173,6 @@ SECTIONS: tuple[SectionSpec, ...] = (
PlotSpec("Expedition Cards", ("avg_expedition_cards",), "cards"), PlotSpec("Expedition Cards", ("avg_expedition_cards",), "cards"),
), ),
), ),
SectionSpec(
"OpenQuality",
"analysis_05_open_quality.png",
(
PlotSpec(
"Bad Open Rate",
("bad_open_rate",),
"rate (%)",
scale=100.0,
fixed_ylim=(0, 100),
),
PlotSpec(
"Weak Open Rate",
("weak_open_rate",),
"rate (%)",
scale=100.0,
fixed_ylim=(0, 100),
),
PlotSpec(
"Bad or Weak Open Rate",
("bad_or_weak_open_rate",),
"rate (%)",
scale=100.0,
fixed_ylim=(0, 100),
),
PlotSpec(
"Good Open Rate",
("good_open_rate",),
"rate (%)",
scale=100.0,
fixed_ylim=(0, 100),
),
PlotSpec("Bad Open per Game", ("bad_open_per_game",), "opens / game"),
PlotSpec(
"Bad or Weak Open per Game",
("bad_or_weak_open_per_game",),
"opens / game",
),
PlotSpec("Opening Play Actions", ("opening_play_actions",), "actions / game"),
PlotSpec("Opening Recoverable p25", ("opening_recoverable_score_p25",), "score"),
),
),
SectionSpec( SectionSpec(
"ExpeditionOutcomes", "ExpeditionOutcomes",
"analysis_06_expedition_outcomes.png", "analysis_06_expedition_outcomes.png",
@@ -188,23 +212,6 @@ SECTIONS: tuple[SectionSpec, ...] = (
PlotSpec("Score per Opened Color", ("score_per_opened_color",), "score / color"), PlotSpec("Score per Opened Color", ("score_per_opened_color",), "score / color"),
), ),
), ),
SectionSpec(
"Calibration",
"analysis_07_calibration.png",
(
PlotSpec(
"First Open Recoverable Score",
(
"first_open_recoverable_score_mean_for_positive_final",
"first_open_recoverable_score_mean_for_negative_final",
),
"score",
),
PlotSpec("Opening Recoverable Mean", ("opening_recoverable_score_mean",), "score"),
PlotSpec("Opening Recoverable p25", ("opening_recoverable_score_p25",), "score"),
PlotSpec("Calibration Gap", ("calibration_gap",), "score"),
),
),
SectionSpec( SectionSpec(
"Traversal", "Traversal",
"analysis_08_traversal.png", "analysis_08_traversal.png",
@@ -311,41 +318,23 @@ SECTIONS: tuple[SectionSpec, ...] = (
), ),
) )
SELECTIVITY_PLOTS: tuple[PlotSpec, ...] = (
PlotSpec("Opened Colors", ("avg_opened_colors",), "colors"),
PlotSpec("5-Color Open Count", ("5_color_open_count",), "games / eval"),
PlotSpec("Opening Recoverable Mean", ("opening_recoverable_score_mean",), "score"),
PlotSpec("Calibration Gap", ("calibration_gap",), "score"),
PlotSpec(
"First Open Recoverable Score",
(
"first_open_recoverable_score_mean_for_positive_final",
"first_open_recoverable_score_mean_for_negative_final",
),
"score",
),
PlotSpec("Opening Play Actions", ("opening_play_actions",), "actions / game"),
)
SUMMARY_EVAL_METRICS: tuple[tuple[str, str, float], ...] = ( SUMMARY_EVAL_METRICS: tuple[tuple[str, str, float], ...] = (
("win_rate0", "win rate (%)", 100.0), ("win_rate0", "win rate (%)", 100.0),
("avg_score_diff0", "avg score diff", 1.0), ("avg_score_diff0", "avg score diff", 1.0),
("avg_score0", "avg score", 1.0), ("avg_score0", "avg score", 1.0),
("play_action_rate", "play rate (%)", 100.0), ("play_action_rate", "play rate (%)", 100.0),
("avg_opened_colors", "opened colors", 1.0), ("avg_opened_colors", "opened colors", 1.0),
("bad_open_rate", "bad open (%)", 100.0),
("score_per_opened_color", "score / opened color", 1.0), ("score_per_opened_color", "score / opened color", 1.0),
("calibration_gap", "calibration gap", 1.0),
("bonus_contribution_per_game", "bonus / game", 1.0), ("bonus_contribution_per_game", "bonus / game", 1.0),
) )
OPPONENT_COLORS: dict[str, str] = { OPPONENT_COLORS: dict[str, str] = {
"noisy_safe": "tab:blue", "heuristic_noisy": "tab:blue",
"passive_discard": "tab:orange", "discard_only": "tab:orange",
"random": "tab:green", "random": "tab:green",
"safe_heuristic": "tab:red", "heuristic_balanced": "tab:red",
"safe_heuristic_loose": "tab:purple", "heuristic_aggressive": "tab:purple",
"safe_heuristic_strict": "tab:brown", "heuristic_cautious": "tab:brown",
} }
TRAVERSAL_COLORS: dict[str, str] = { TRAVERSAL_COLORS: dict[str, str] = {
@@ -428,7 +417,14 @@ def plot_section(
if spec.kind == "train": if spec.kind == "train":
plotted = _plot_train_spec(ax, rows, spec, smoothing_window=smoothing_window) plotted = _plot_train_spec(ax, rows, spec, smoothing_window=smoothing_window)
else: else:
plotted = _plot_eval_spec(ax, rows, opponents, spec, smoothing_window=smoothing_window) filtered_opponents = (
[o for o in opponents if o in spec.opponents]
if spec.opponents is not None
else opponents
)
plotted = _plot_eval_spec(
ax, rows, filtered_opponents, spec, smoothing_window=smoothing_window
)
_finish_axis( _finish_axis(
ax, spec.title, ylabel=spec.ylabel, plotted=plotted, fixed_ylim=spec.fixed_ylim ax, spec.title, ylabel=spec.ylabel, plotted=plotted, fixed_ylim=spec.fixed_ylim
) )
@@ -539,12 +535,6 @@ def analyze_run(
if plot_section(rows, section, path, smoothing_window=smoothing_window): if plot_section(rows, section, path, smoothing_window=smoothing_window):
written.append(path) written.append(path)
selectivity_path = output_dir / _with_filename_suffix(
"analysis_09_selectivity.png", filename_suffix
)
if plot_selectivity(rows, selectivity_path, smoothing_window=smoothing_window):
written.append(selectivity_path)
final_eval_path = output_dir / _with_filename_suffix( final_eval_path = output_dir / _with_filename_suffix(
"analysis_final_eval_summary.png", filename_suffix "analysis_final_eval_summary.png", filename_suffix
) )
@@ -566,45 +556,6 @@ def _with_filename_suffix(filename: str, suffix: str) -> str:
return f"{path.stem}{suffix}{path.suffix}" return f"{path.stem}{suffix}{path.suffix}"
def plot_selectivity(
rows: list[dict[str, Any]],
output: Path,
*,
smoothing_window: int,
) -> bool:
import matplotlib.pyplot as plt
opponents = opponent_names(rows)
if not opponents:
return False
fig, axes = plt.subplots(3, 2, figsize=(16, 12), squeeze=False)
axes_flat = list(axes.flat)
plotted_any = False
for ax, spec in zip(axes_flat, SELECTIVITY_PLOTS, strict=True):
plotted = _plot_eval_spec(ax, rows, opponents, spec, smoothing_window=smoothing_window)
_finish_axis(ax, spec.title, ylabel=spec.ylabel, plotted=plotted)
plotted_any = plotted_any or plotted
handles, labels = _legend_items(axes_flat)
if handles:
fig.legend(handles, labels, loc="upper center", ncols=min(len(labels), 6), fontsize="small")
suffix = f" ({smoothing_window}-iter moving average)" if smoothing_window > 1 else ""
fig.suptitle(
f"Lost Cities Deep CFR selectivity metrics{suffix}",
fontsize=14,
fontweight="bold",
)
fig.tight_layout(rect=(0, 0, 1, 0.95))
if not plotted_any:
plt.close(fig)
return False
fig.savefig(output, dpi=150)
plt.close(fig)
return True
def _all_eval_metrics() -> set[str]: def _all_eval_metrics() -> set[str]:
metrics: set[str] = set() metrics: set[str] = set()
for section in SECTIONS: for section in SECTIONS:
@@ -143,16 +143,16 @@ class TraversalConfig(StrictModel):
@field_validator("cutoff_rollout_policy") @field_validator("cutoff_rollout_policy")
@classmethod @classmethod
def _validate_cutoff_rollout_policy(cls, value: str) -> str: def _validate_cutoff_rollout_policy(cls, value: str) -> str:
if value not in {"random", "safe_heuristic"}: if value not in {"random", "heuristic_balanced"}:
raise ValueError("must be 'random' or 'safe_heuristic'") raise ValueError("must be 'random' or 'heuristic_balanced'")
return value return value
@field_validator("opponent_policy") @field_validator("opponent_policy")
@classmethod @classmethod
def _validate_opponent_policy(cls, value: str) -> str: def _validate_opponent_policy(cls, value: str) -> str:
if value not in {"network", "safe_heuristic", "self_play_league", "average_strategy"}: if value not in {"network", "heuristic_balanced", "self_play_league", "average_strategy"}:
raise ValueError( raise ValueError(
"must be 'network', 'safe_heuristic', 'self_play_league', or 'average_strategy'" "must be 'network', 'heuristic_balanced', 'self_play_league', or 'average_strategy'"
) )
return value return value
@@ -294,13 +294,35 @@ class CheckpointConfig(StrictModel):
class EvaluationConfig(StrictModel): class EvaluationConfig(StrictModel):
eval_every: int = 50 eval_every: int = 50
games: int = 10 games: int = 10
opponents: tuple[str, ...] = ("random", "safe_heuristic") opponents: tuple[str, ...] = ("random", "heuristic_balanced")
extended_eval_every: int = 0
extended_opponents: tuple[str, ...] = ()
max_steps: int = 10_000 max_steps: int = 10_000
on_max_steps: str = "score_diff" on_max_steps: str = "score_diff"
batch_size: int = 64 batch_size: int = 64
device: str = "trainer" device: str = "trainer"
num_workers: int = 4 num_workers: int = 4
def opponents_for_iteration(self, iteration: int) -> tuple[str, ...]:
"""Return the opponent set to evaluate against at ``iteration``.
- Always returns ``opponents`` when ``iteration % eval_every == 0``.
- When ``extended_eval_every > 0`` and ``iteration`` is also a multiple
of ``extended_eval_every``, appends ``extended_opponents`` (de-duplicated)
to the core list.
- Returns ``()`` when no eval is scheduled this iteration.
"""
if self.eval_every <= 0 or iteration <= 0:
return ()
if iteration % self.eval_every != 0:
return ()
result = list(self.opponents)
if self.extended_eval_every > 0 and iteration % self.extended_eval_every == 0:
for opponent in self.extended_opponents:
if opponent not in result:
result.append(opponent)
return tuple(result)
@field_validator("on_max_steps") @field_validator("on_max_steps")
@classmethod @classmethod
def _validate_on_max_steps(cls, value: str) -> str: def _validate_on_max_steps(cls, value: str) -> str:
@@ -6,7 +6,7 @@ import numpy as np
import torch import torch
from torch import nn from torch import nn
from coolrl_lost_cities.games.classic.bots import SafeHeuristicBot from coolrl_lost_cities.games.classic.bots import HeuristicBot
from coolrl_lost_cities.games.classic.deep_cfr.encoding import encode_info_state, input_dim from coolrl_lost_cities.games.classic.deep_cfr.encoding import encode_info_state, input_dim
from coolrl_lost_cities.games.classic.deep_cfr.networks import DeepCFRMLP from coolrl_lost_cities.games.classic.deep_cfr.networks import DeepCFRMLP
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig, classic_config from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig, classic_config
@@ -18,7 +18,7 @@ class ImitationMetrics:
loss: float loss: float
def collect_safe_heuristic_samples( def collect_heuristic_samples(
config: LostCitiesConfig | None = None, config: LostCitiesConfig | None = None,
*, *,
games: int = 4, games: int = 4,
@@ -28,7 +28,7 @@ def collect_safe_heuristic_samples(
game_config = config or classic_config(seed=seed) game_config = config or classic_config(seed=seed)
probe = GameState.new_game(game_config, seed=seed) probe = GameState.new_game(game_config, seed=seed)
action_size = 2 * probe.config.hand_size + 1 + probe.config.n_colors action_size = 2 * probe.config.hand_size + 1 + probe.config.n_colors
bot = SafeHeuristicBot() bot = HeuristicBot()
infos: list[np.ndarray] = [] infos: list[np.ndarray] = []
targets: list[np.ndarray] = [] targets: list[np.ndarray] = []
masks: list[np.ndarray] = [] masks: list[np.ndarray] = []
@@ -67,7 +67,7 @@ def pretrain_strategy_network(
learning_rate: float = 1.0e-3, learning_rate: float = 1.0e-3,
device: torch.device | str = "cpu", device: torch.device | str = "cpu",
) -> ImitationMetrics: ) -> ImitationMetrics:
x_np, y_np, legal_np = collect_safe_heuristic_samples(config, games=games, seed=seed) x_np, y_np, legal_np = collect_heuristic_samples(config, games=games, seed=seed)
device = torch.device(device) device = torch.device(device)
strategy_network.to(device) strategy_network.to(device)
strategy_network.train() strategy_network.train()
@@ -894,17 +894,15 @@ class DeepCFRTrainer:
self.tracker.log_event(_format_iteration_summary(metrics, data)) self.tracker.log_event(_format_iteration_summary(metrics, data))
def _evaluate(self, iteration: int) -> dict[str, float | int]: def _evaluate(self, iteration: int) -> dict[str, float | int]:
if ( opponents = self.config.evaluation.opponents_for_iteration(iteration)
self.config.evaluation.eval_every <= 0 if not opponents:
or iteration % self.config.evaluation.eval_every != 0
):
return {} return {}
results: dict[str, float | int] = {} results: dict[str, float | int] = {}
eval_device = self._evaluation_device() eval_device = self._evaluation_device()
if self.config.evaluation.resolved_num_workers(len(self.config.evaluation.opponents)) > 1: if self.config.evaluation.resolved_num_workers(len(opponents)) > 1:
return self._evaluate_parallel(iteration, eval_device) return self._evaluate_parallel(iteration, eval_device, opponents)
eval_network = self._evaluation_network(eval_device) eval_network = self._evaluation_network(eval_device)
for opponent in self.config.evaluation.opponents: for opponent in opponents:
result = evaluate_strategy_network( result = evaluate_strategy_network(
eval_network, eval_network,
self.game_config, self.game_config,
@@ -925,8 +923,8 @@ class DeepCFRTrainer:
self, self,
iteration: int, iteration: int,
eval_device: torch.device, eval_device: torch.device,
opponents: tuple[str, ...],
) -> dict[str, float | int]: ) -> dict[str, float | int]:
opponents = self.config.evaluation.opponents
max_workers = self.config.evaluation.resolved_num_workers(len(opponents)) max_workers = self.config.evaluation.resolved_num_workers(len(opponents))
self.tracker.log_event( self.tracker.log_event(
f"Evaluation multiprocessing enabled iteration={iteration} " f"Evaluation multiprocessing enabled iteration={iteration} "
@@ -7,7 +7,7 @@ from libc.stdlib cimport free, malloc
import numpy as np import numpy as np
import torch import torch
from coolrl_lost_cities.games.classic.bots import SafeHeuristicBot from coolrl_lost_cities.games.classic.bots import HeuristicBot
from coolrl_lost_cities.games.classic.deep_cfr.cfr_math cimport regret_matching_c from coolrl_lost_cities.games.classic.deep_cfr.cfr_math cimport regret_matching_c
from coolrl_lost_cities.games.classic.deep_cfr.encoding cimport ( from coolrl_lost_cities.games.classic.deep_cfr.encoding cimport (
_encode_info_state_with_flags_c, _encode_info_state_with_flags_c,
@@ -70,8 +70,8 @@ cdef class CythonDeepCFRTraverser:
cdef object device cdef object device
cdef object encoding cdef object encoding
cdef object league_advantage_networks cdef object league_advantage_networks
cdef object safe_heuristic_rollout_bot cdef object heuristic_rollout_bot
cdef object safe_heuristic_opponent_bot cdef object heuristic_opponent_bot
cdef int action_size cdef int action_size
cdef int input_dim cdef int input_dim
cdef float epsilon cdef float epsilon
@@ -185,15 +185,15 @@ cdef class CythonDeepCFRTraverser:
raise ValueError("cutoff_value_mode must be 'score_diff' or 'random_rollout'") raise ValueError("cutoff_value_mode must be 'score_diff' or 'random_rollout'")
self.cutoff_random_rollout = cutoff_value_mode == "random_rollout" self.cutoff_random_rollout = cutoff_value_mode == "random_rollout"
self.cutoff_rollouts = max(0, cutoff_rollouts) self.cutoff_rollouts = max(0, cutoff_rollouts)
if cutoff_rollout_policy not in {"random", "safe_heuristic"}: if cutoff_rollout_policy not in {"random", "heuristic_balanced"}:
raise ValueError("cutoff_rollout_policy must be 'random' or 'safe_heuristic'") raise ValueError("cutoff_rollout_policy must be 'random' or 'heuristic_balanced'")
self.cutoff_rollout_max_steps = max(1, cutoff_rollout_max_steps) self.cutoff_rollout_max_steps = max(1, cutoff_rollout_max_steps)
self.safe_heuristic_rollout_bot = ( self.heuristic_rollout_bot = (
SafeHeuristicBot() if cutoff_rollout_policy == "safe_heuristic" else None HeuristicBot() if cutoff_rollout_policy == "heuristic_balanced" else None
) )
if opponent_policy == "network": if opponent_policy == "network":
self.opponent_policy_id = 0 self.opponent_policy_id = 0
elif opponent_policy == "safe_heuristic": elif opponent_policy == "heuristic_balanced":
self.opponent_policy_id = 1 self.opponent_policy_id = 1
elif opponent_policy == "self_play_league": elif opponent_policy == "self_play_league":
self.opponent_policy_id = 2 self.opponent_policy_id = 2
@@ -205,7 +205,7 @@ cdef class CythonDeepCFRTraverser:
) )
else: else:
raise ValueError( raise ValueError(
"opponent_policy must be 'network', 'safe_heuristic', 'self_play_league', or 'average_strategy'" "opponent_policy must be 'network', 'heuristic_balanced', 'self_play_league', or 'average_strategy'"
) )
if all_negative_fallback == "uniform": if all_negative_fallback == "uniform":
self.all_negative_fallback_id = 0 self.all_negative_fallback_id = 0
@@ -225,8 +225,8 @@ cdef class CythonDeepCFRTraverser:
self.self_play_recent_window = max(0, self_play_recent_window) self.self_play_recent_window = max(0, self_play_recent_window)
self.active_self_play_bucket = 0 self.active_self_play_bucket = 0
self.active_self_play_networks = None self.active_self_play_networks = None
self.safe_heuristic_opponent_bot = ( self.heuristic_opponent_bot = (
SafeHeuristicBot() HeuristicBot()
if self.opponent_policy_id == 1 or self.self_play_anchor_probability > 0.0 if self.opponent_policy_id == 1 or self.self_play_anchor_probability > 0.0
else None else None
) )
@@ -630,9 +630,9 @@ cdef class CythonDeepCFRTraverser:
if player == traverser or self.opponent_policy_id == 0: if player == traverser or self.opponent_policy_id == 0:
return -1 return -1
if self.opponent_policy_id == 1: if self.opponent_policy_id == 1:
if self.safe_heuristic_opponent_bot is None: if self.heuristic_opponent_bot is None:
self.safe_heuristic_opponent_bot = SafeHeuristicBot() self.heuristic_opponent_bot = HeuristicBot()
return int(self.safe_heuristic_opponent_bot.act(state)) return int(self.heuristic_opponent_bot.act(state))
if self.opponent_policy_id == 3: if self.opponent_policy_id == 3:
self._policy_from_strategy_network(state, player, legal, policy) self._policy_from_strategy_network(state, player, legal, policy)
for i in range(self.action_size): for i in range(self.action_size):
@@ -649,9 +649,9 @@ cdef class CythonDeepCFRTraverser:
if bucket == 0: if bucket == 0:
return -1 return -1
if bucket == 3: if bucket == 3:
if self.safe_heuristic_opponent_bot is None: if self.heuristic_opponent_bot is None:
self.safe_heuristic_opponent_bot = SafeHeuristicBot() self.heuristic_opponent_bot = HeuristicBot()
return int(self.safe_heuristic_opponent_bot.act(state)) return int(self.heuristic_opponent_bot.act(state))
networks = self.active_self_play_networks networks = self.active_self_play_networks
if networks is None: if networks is None:
return -1 return -1
@@ -839,8 +839,8 @@ cdef class CythonDeepCFRTraverser:
if swapped_indices == NULL: if swapped_indices == NULL:
raise MemoryError() raise MemoryError()
while not state.terminal and steps < self.cutoff_rollout_max_steps: while not state.terminal and steps < self.cutoff_rollout_max_steps:
if self.safe_heuristic_rollout_bot is not None: if self.heuristic_rollout_bot is not None:
local_action = int(self.safe_heuristic_rollout_bot.act(state)) local_action = int(self.heuristic_rollout_bot.act(state))
unified_action = self._to_unified_action_c(state, local_action) unified_action = self._to_unified_action_c(state, local_action)
else: else:
count = state._unified_legal_actions_c(actions) count = state._unified_legal_actions_c(actions)
@@ -277,7 +277,7 @@ def evaluate_policy(
def main(argv: list[str] | None = None) -> None: def main(argv: list[str] | None = None) -> None:
parser = argparse.ArgumentParser(description="Evaluate Lost Cities classic bots.") parser = argparse.ArgumentParser(description="Evaluate Lost Cities classic bots.")
parser.add_argument("--bot0", default="safe-heuristic", choices=available_bot_names()) parser.add_argument("--bot0", default="heuristic-balanced", choices=available_bot_names())
parser.add_argument("--bot1", default="random", choices=available_bot_names()) parser.add_argument("--bot1", default="random", choices=available_bot_names())
parser.add_argument("--games", type=int, default=100) parser.add_argument("--games", type=int, default=100)
parser.add_argument("--seed", type=int, default=1) parser.add_argument("--seed", type=int, default=1)
+21 -21
View File
@@ -1,9 +1,9 @@
from coolrl_lost_cities.games.classic.game import Card, GameState, LostCitiesConfig from coolrl_lost_cities.games.classic.game import Card, GameState, LostCitiesConfig
from coolrl_lost_cities.games.classic.bots import ( from coolrl_lost_cities.games.classic.bots import (
HeuristicBot,
LostCitiesPolicy, LostCitiesPolicy,
RandomBot, RandomBot,
SafeHeuristicBot,
) )
from coolrl_lost_cities.games.classic.bots.heuristic import draw_from_discard_action from coolrl_lost_cities.games.classic.bots.heuristic import draw_from_discard_action
from coolrl_lost_cities.games.classic.evaluation import play_game_for_evaluation from coolrl_lost_cities.games.classic.evaluation import play_game_for_evaluation
@@ -19,13 +19,13 @@ def _expeditions(config: LostCitiesConfig) -> list[list[list[Card]]]:
def test_builtin_bots_implement_lost_cities_policy() -> None: def test_builtin_bots_implement_lost_cities_policy() -> None:
assert isinstance(RandomBot(1), LostCitiesPolicy) assert isinstance(RandomBot(1), LostCitiesPolicy)
assert isinstance(SafeHeuristicBot(), LostCitiesPolicy) assert isinstance(HeuristicBot(), LostCitiesPolicy)
def test_safe_heuristic_mirror_match_finishes() -> None: def test_heuristic_mirror_match_finishes() -> None:
state, result = play_game_for_evaluation( state, result = play_game_for_evaluation(
SafeHeuristicBot(), HeuristicBot(),
SafeHeuristicBot(), HeuristicBot(),
LostCitiesConfig(n_colors=3, n_ranks=5, n_handshakes=1, hand_size=5), LostCitiesConfig(n_colors=3, n_ranks=5, n_handshakes=1, hand_size=5),
seed=2000, seed=2000,
max_steps=200, max_steps=200,
@@ -34,9 +34,9 @@ def test_safe_heuristic_mirror_match_finishes() -> None:
assert result.timed_out is False assert result.timed_out is False
def test_safe_heuristic_opponent_value_ignores_hidden_hand() -> None: def test_heuristic_opponent_value_ignores_hidden_hand() -> None:
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3) config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
bot = SafeHeuristicBot() bot = HeuristicBot()
discard_card = Card(color=0, rank=6) discard_card = Card(color=0, rank=6)
expeditions_a = _expeditions(config) expeditions_a = _expeditions(config)
@@ -76,9 +76,9 @@ def test_safe_heuristic_opponent_value_ignores_hidden_hand() -> None:
assert value_a == value_b assert value_a == value_b
def test_safe_heuristic_started_expedition_value_ignores_invalid_lower_followup() -> None: def test_heuristic_started_expedition_value_ignores_invalid_lower_followup() -> None:
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3) config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
bot = SafeHeuristicBot() bot = HeuristicBot()
high_card = Card(color=0, rank=8) high_card = Card(color=0, rank=8)
base_expeditions = _expeditions(config) base_expeditions = _expeditions(config)
@@ -115,9 +115,9 @@ def test_safe_heuristic_started_expedition_value_ignores_invalid_lower_followup(
assert lower_followup_value == base_value assert lower_followup_value == base_value
def test_safe_heuristic_draws_playable_discard_instead_of_deck() -> None: def test_heuristic_draws_playable_discard_instead_of_deck() -> None:
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3) config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
bot = SafeHeuristicBot() bot = HeuristicBot()
expeditions = _expeditions(config) expeditions = _expeditions(config)
expeditions[0][0] = [Card(color=0, rank=4)] expeditions[0][0] = [Card(color=0, rank=4)]
@@ -132,9 +132,9 @@ def test_safe_heuristic_draws_playable_discard_instead_of_deck() -> None:
assert bot._act_draw(state) == draw_from_discard_action(0) assert bot._act_draw(state) == draw_from_discard_action(0)
def test_safe_heuristic_can_draw_discard_to_deny_opponent_when_losing() -> None: def test_heuristic_can_draw_discard_to_deny_opponent_when_losing() -> None:
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=4) config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=4)
bot = SafeHeuristicBot() bot = HeuristicBot()
expeditions = _expeditions(config) expeditions = _expeditions(config)
expeditions[0][1] = [Card(color=1, rank=8)] expeditions[0][1] = [Card(color=1, rank=8)]
@@ -158,9 +158,9 @@ def test_safe_heuristic_can_draw_discard_to_deny_opponent_when_losing() -> None:
assert bot._act_draw(state) == draw_from_discard_action(0) assert bot._act_draw(state) == draw_from_discard_action(0)
def test_safe_heuristic_classic_self_play_opens_expeditions() -> None: def test_heuristic_classic_self_play_opens_expeditions() -> None:
state = GameState.new_game(LostCitiesConfig(), seed=1) state = GameState.new_game(LostCitiesConfig(), seed=1)
bot = SafeHeuristicBot() bot = HeuristicBot()
player0_actions: list[int] = [] player0_actions: list[int] = []
for _ in range(60): for _ in range(60):
@@ -182,9 +182,9 @@ def test_safe_heuristic_classic_self_play_opens_expeditions() -> None:
assert any(state.expeditions[0][color] for color in range(state.config.n_colors)) assert any(state.expeditions[0][color] for color in range(state.config.n_colors))
def test_safe_heuristic_avoids_opening_weak_fifth_color() -> None: def test_heuristic_avoids_opening_weak_fifth_color() -> None:
config = LostCitiesConfig(n_colors=5, n_ranks=8, hand_size=8) config = LostCitiesConfig(n_colors=5, n_ranks=8, hand_size=8)
bot = SafeHeuristicBot() bot = HeuristicBot()
expeditions = _expeditions(config) expeditions = _expeditions(config)
expeditions[0][0] = [Card(color=0, rank=4)] expeditions[0][0] = [Card(color=0, rank=4)]
expeditions[0][1] = [Card(color=1, rank=4)] expeditions[0][1] = [Card(color=1, rank=4)]
@@ -211,9 +211,9 @@ def test_safe_heuristic_avoids_opening_weak_fifth_color() -> None:
) )
def test_safe_heuristic_prefers_followup_on_started_expedition() -> None: def test_heuristic_prefers_followup_on_started_expedition() -> None:
config = LostCitiesConfig(n_colors=3, n_ranks=8, hand_size=5) config = LostCitiesConfig(n_colors=3, n_ranks=8, hand_size=5)
bot = SafeHeuristicBot() bot = HeuristicBot()
expeditions = _expeditions(config) expeditions = _expeditions(config)
expeditions[0][0] = [Card(color=0, rank=4)] expeditions[0][0] = [Card(color=0, rank=4)]
state = make_state( state = make_state(
@@ -233,9 +233,9 @@ def test_safe_heuristic_prefers_followup_on_started_expedition() -> None:
assert chosen.color == 0 assert chosen.color == 0
def test_safe_heuristic_avoids_unopened_discard_draw_after_four_opens() -> None: def test_heuristic_avoids_unopened_discard_draw_after_four_opens() -> None:
config = LostCitiesConfig(n_colors=5, n_ranks=8, hand_size=8) config = LostCitiesConfig(n_colors=5, n_ranks=8, hand_size=8)
bot = SafeHeuristicBot() bot = HeuristicBot()
expeditions = _expeditions(config) expeditions = _expeditions(config)
expeditions[0][0] = [Card(color=0, rank=4)] expeditions[0][0] = [Card(color=0, rank=4)]
expeditions[0][1] = [Card(color=1, rank=4)] expeditions[0][1] = [Card(color=1, rank=4)]
@@ -3,13 +3,13 @@ from __future__ import annotations
from coolrl_lost_cities.games.classic.game import LostCitiesConfig from coolrl_lost_cities.games.classic.game import LostCitiesConfig
from coolrl_lost_cities.games.classic.deep_cfr.imitation import ( from coolrl_lost_cities.games.classic.deep_cfr.imitation import (
collect_safe_heuristic_samples, collect_heuristic_samples,
new_pretrained_strategy_network, new_pretrained_strategy_network,
) )
def test_collect_safe_heuristic_samples_shapes() -> None: def test_collect_heuristic_samples_shapes() -> None:
x, y, legal = collect_safe_heuristic_samples(LostCitiesConfig(seed=61), games=1, seed=61) x, y, legal = collect_heuristic_samples(LostCitiesConfig(seed=61), games=1, seed=61)
assert len(x) == len(y) == len(legal) assert len(x) == len(y) == len(legal)
assert x.ndim == 2 assert x.ndim == 2
@@ -1494,3 +1494,83 @@ def test_interleaved_regret_matching_no_fallback_unchanged_by_mode() -> None:
assert fallback_a is False assert fallback_a is False
assert np.allclose(policy_uniform, policy_argmax) assert np.allclose(policy_uniform, policy_argmax)
assert np.allclose(policy_uniform.sum(), 1.0) assert np.allclose(policy_uniform.sum(), 1.0)
def test_evaluation_opponents_for_iteration_core_only_when_extended_disabled() -> None:
from coolrl_lost_cities.games.classic.deep_cfr.config import EvaluationConfig
cfg = EvaluationConfig(
eval_every=5,
opponents=("random", "discard_only", "heuristic_cautious"),
extended_eval_every=0,
extended_opponents=("heuristic_balanced",),
)
assert cfg.opponents_for_iteration(0) == ()
assert cfg.opponents_for_iteration(3) == ()
assert cfg.opponents_for_iteration(5) == (
"random",
"discard_only",
"heuristic_cautious",
)
assert cfg.opponents_for_iteration(50) == (
"random",
"discard_only",
"heuristic_cautious",
)
def test_evaluation_opponents_for_iteration_extends_on_extended_cadence() -> None:
from coolrl_lost_cities.games.classic.deep_cfr.config import EvaluationConfig
cfg = EvaluationConfig(
eval_every=5,
opponents=("random", "discard_only", "heuristic_cautious"),
extended_eval_every=50,
extended_opponents=(
"heuristic_balanced",
"heuristic_aggressive",
"heuristic_noisy",
),
)
assert cfg.opponents_for_iteration(5) == (
"random",
"discard_only",
"heuristic_cautious",
)
assert cfg.opponents_for_iteration(45) == (
"random",
"discard_only",
"heuristic_cautious",
)
assert cfg.opponents_for_iteration(50) == (
"random",
"discard_only",
"heuristic_cautious",
"heuristic_balanced",
"heuristic_aggressive",
"heuristic_noisy",
)
assert cfg.opponents_for_iteration(100) == (
"random",
"discard_only",
"heuristic_cautious",
"heuristic_balanced",
"heuristic_aggressive",
"heuristic_noisy",
)
def test_evaluation_opponents_for_iteration_dedupes_overlap() -> None:
from coolrl_lost_cities.games.classic.deep_cfr.config import EvaluationConfig
cfg = EvaluationConfig(
eval_every=5,
opponents=("random", "heuristic_cautious"),
extended_eval_every=10,
extended_opponents=("heuristic_cautious", "heuristic_balanced"),
)
assert cfg.opponents_for_iteration(10) == (
"random",
"heuristic_cautious",
"heuristic_balanced",
)
+3 -3
View File
@@ -13,7 +13,7 @@ def test_play_game_for_evaluation_finishes_small_match() -> None:
state, result = play_game_for_evaluation( state, result = play_game_for_evaluation(
build_bot("random", seed=1), build_bot("random", seed=1),
build_bot("passive-discard", seed=2), build_bot("discard-only", seed=2),
config, config,
seed=3, seed=3,
max_steps=200, max_steps=200,
@@ -30,7 +30,7 @@ def test_play_match_alternates_seats_and_reports_rates() -> None:
result = play_match( result = play_match(
make_policy_factory("random"), make_policy_factory("random"),
make_policy_factory("passive-discard"), make_policy_factory("discard-only"),
config, config,
games=4, games=4,
seed=10, seed=10,
@@ -50,7 +50,7 @@ def test_evaluation_cli_smoke_json(capsys) -> None:
"--bot0", "--bot0",
"random", "random",
"--bot1", "--bot1",
"passive-discard", "discard-only",
"--games", "--games",
"2", "2",
"--seed", "--seed",
+5 -5
View File
@@ -43,10 +43,10 @@ def test_classic_package_exports_bot_registry_helpers() -> None:
def test_classic_bot_registry_accepts_reproduction_opponent_names() -> None: def test_classic_bot_registry_accepts_reproduction_opponent_names() -> None:
for name in [ for name in [
"random", "random",
"passive_discard", "discard_only",
"safe_heuristic", "heuristic_balanced",
"safe_heuristic_loose", "heuristic_aggressive",
"safe_heuristic_strict", "heuristic_cautious",
"noisy_safe", "heuristic_noisy",
]: ]:
assert isinstance(classic.build_bot(name, seed=1), classic.LostCitiesPolicy) assert isinstance(classic.build_bot(name, seed=1), classic.LostCitiesPolicy)
+2 -2
View File
@@ -11,7 +11,7 @@ def test_gui_argparser_accepts_classic_options() -> None:
"--mode", "--mode",
"pvc", "pvc",
"--bot", "--bot",
"safe-heuristic", "heuristic-balanced",
"--seed", "--seed",
"7", "7",
"--width", "--width",
@@ -22,7 +22,7 @@ def test_gui_argparser_accepts_classic_options() -> None:
) )
assert args.mode == "pvc" assert args.mode == "pvc"
assert args.bot == "safe-heuristic" assert args.bot == "heuristic-balanced"
assert args.seed == 7 assert args.seed == 7
assert args.width == 1024 assert args.width == 1024
assert args.height == 768 assert args.height == 768
@@ -3,19 +3,19 @@ from __future__ import annotations
import pytest import pytest
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig
from coolrl_lost_cities.games.classic.bots.heuristic import SafeHeuristicBot from coolrl_lost_cities.games.classic.bots.heuristic import HeuristicBot
from coolrl_lost_cities.games.classic.bots.heuristic_py import ( from coolrl_lost_cities.games.classic.bots.heuristic_py import (
SafeHeuristicBot as PythonSafeHeuristicBot, HeuristicBot as PythonHeuristicBot,
) )
from coolrl_lost_cities.games.classic.bots.registry import ( from coolrl_lost_cities.games.classic.bots.registry import (
LOOSE_SAFE_HEURISTIC_PARAMS, AGGRESSIVE_HEURISTIC_PARAMS,
STRICT_SAFE_HEURISTIC_PARAMS, CAUTIOUS_HEURISTIC_PARAMS,
) )
VARIANTS = ( VARIANTS = (
("default", None), ("default", None),
("loose", LOOSE_SAFE_HEURISTIC_PARAMS), ("loose", AGGRESSIVE_HEURISTIC_PARAMS),
("strict", STRICT_SAFE_HEURISTIC_PARAMS), ("strict", CAUTIOUS_HEURISTIC_PARAMS),
) )
CONFIGS = ( CONFIGS = (
@@ -28,14 +28,14 @@ CONFIGS = (
@pytest.mark.parametrize(("variant_name", "params"), VARIANTS) @pytest.mark.parametrize(("variant_name", "params"), VARIANTS)
@pytest.mark.parametrize("config", CONFIGS) @pytest.mark.parametrize("config", CONFIGS)
@pytest.mark.parametrize("seed", range(2)) @pytest.mark.parametrize("seed", range(2))
def test_cython_safe_heuristic_matches_python_action_sequence( def test_cython_heuristic_matches_python_action_sequence(
variant_name: str, variant_name: str,
params, params,
config: LostCitiesConfig, config: LostCitiesConfig,
seed: int, seed: int,
) -> None: ) -> None:
py_bot = PythonSafeHeuristicBot(params) py_bot = PythonHeuristicBot(params)
cy_bot = SafeHeuristicBot(params) cy_bot = HeuristicBot(params)
py_state = GameState.new_game(config, seed=seed) py_state = GameState.new_game(config, seed=seed)
cy_state = GameState.new_game(config, seed=seed) cy_state = GameState.new_game(config, seed=seed)