Rename bot family, curate analyze plots, tier evaluation cadence
Three coordinated hygiene changes; none target the diagnosed
selection-bias bottleneck. They make the codebase honestly reflect the
pure-self-play stance and reduce dashboard noise.
Bot rename (drop the unhelpful safe_ prefix; suffixes describe behaviour):
- safe_heuristic_loose -> heuristic_aggressive
- safe_heuristic -> heuristic_balanced
- safe_heuristic_strict -> heuristic_cautious
- noisy_safe -> heuristic_noisy
- passive_discard -> discard_only
Class renames in bots/: SafeHeuristicBot -> HeuristicBot,
SafeHeuristicParams -> HeuristicParams, PassiveDiscardBot -> DiscardOnlyBot,
plus loose/strict parameter constants. Backwards compatibility was dropped
intentionally per user instruction; no aliases. Active configs, docs,
scripts, tests updated. Archive directories (configs/archive,
docs/archive, runs/archive) left intact and may still reference old
names per their read-only policy. The src/.../bots/passive.py module was
renamed to discard_only.py via git mv.
Analyze plot curation (deep_cfr/analyze.py):
- Added analysis_00_core.png as the canonical daily dashboard with 10
heuristic-free metrics (loss/{advantage,strategy}; vs heuristic_cautious:
avg_score_diff0, win_rate0, avg_opened_colors, positive_expedition_rate,
bonus_expedition_rate, score_per_opened_color, policy_entropy; vs random:
win_rate0).
- Removed analysis_05_open_quality.png (bad/weak/good open rates,
recoverable score) and analysis_07_calibration.png (calibration gap,
recoverable mean) - both relied on the heuristic recoverable_score
classifier already dropped from inputs.
- Removed SELECTIVITY_PLOTS and plot_selectivity (heuristic-laden).
- SUMMARY_EVAL_METRICS no longer includes bad_open_rate or
calibration_gap.
- PlotSpec gained an opponents allowlist so the new core section can pin
a specific opponent per panel without restructuring plot_section.
Tiered evaluation cadence (EvaluationConfig):
- Added extended_opponents and extended_eval_every (default 0 = disabled).
- opponents_for_iteration(iteration) returns the core list every
eval_every and appends extended_opponents (de-duplicated) when
iteration is also a multiple of extended_eval_every.
- default.yaml now uses 3 core opponents (random, discard_only,
heuristic_cautious) every 5 iterations and 3 extended opponents
(heuristic_balanced, heuristic_aggressive, heuristic_noisy) every 50
iterations. random is the floor sanity. discard_only is the
zero-pit detector / absolute-score reference (its score is always 0,
so eval/discard_only/avg_score_diff0 directly equals the model's raw
average score). heuristic_cautious is the ceiling and the
archive-comparable benchmark used in the prior diagnostic sections.
Net eval cost reduction: roughly 50% (3 opponents x every 5 iter, plus
6 opponents x every 50 iter, vs the prior 6 x every 5).
Documented in docs/plans/deep-cfr-selectivity.md section 9.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -7,7 +7,7 @@ The current implementation starts with the classic two-player card game:
|
|||||||
- classic 5-expedition rules by default
|
- classic 5-expedition rules by default
|
||||||
- Python/Cython game engine
|
- Python/Cython game engine
|
||||||
- env wrapper
|
- env wrapper
|
||||||
- random, passive-discard, and safe-heuristic bots
|
- random, discard-only, and safe-heuristic bots
|
||||||
- core rule, scoring, mask, env, canonical-state, bot, and GUI smoke tests
|
- core rule, scoring, mask, env, canonical-state, bot, and GUI smoke tests
|
||||||
|
|
||||||
Training code, Deep CFR, learned-policy evaluation, GUI, and web client are
|
Training code, Deep CFR, learned-policy evaluation, GUI, and web client are
|
||||||
|
|||||||
@@ -92,11 +92,13 @@ evaluation:
|
|||||||
games: 100
|
games: 100
|
||||||
opponents:
|
opponents:
|
||||||
- random
|
- random
|
||||||
- passive_discard
|
- discard_only
|
||||||
- safe_heuristic
|
- heuristic_cautious
|
||||||
- safe_heuristic_loose
|
extended_eval_every: 50
|
||||||
- safe_heuristic_strict
|
extended_opponents:
|
||||||
- noisy_safe
|
- heuristic_balanced
|
||||||
|
- heuristic_aggressive
|
||||||
|
- heuristic_noisy
|
||||||
max_steps: 10000
|
max_steps: 10000
|
||||||
on_max_steps: score_diff
|
on_max_steps: score_diff
|
||||||
batch_size: 64
|
batch_size: 64
|
||||||
|
|||||||
@@ -89,11 +89,11 @@ evaluation:
|
|||||||
games: 100
|
games: 100
|
||||||
opponents:
|
opponents:
|
||||||
- random
|
- random
|
||||||
- passive_discard
|
- discard_only
|
||||||
- safe_heuristic
|
- heuristic_balanced
|
||||||
- safe_heuristic_loose
|
- heuristic_aggressive
|
||||||
- safe_heuristic_strict
|
- heuristic_cautious
|
||||||
- noisy_safe
|
- heuristic_noisy
|
||||||
max_steps: 10000
|
max_steps: 10000
|
||||||
on_max_steps: score_diff
|
on_max_steps: score_diff
|
||||||
batch_size: 64
|
batch_size: 64
|
||||||
|
|||||||
+10
-10
@@ -182,7 +182,7 @@ eval_<opponent>_<metric>
|
|||||||
|
|
||||||
For that run, iteration 75 had `evaluation_seconds = 18.29s`. Evaluation was
|
For that run, iteration 75 had `evaluation_seconds = 18.29s`. Evaluation was
|
||||||
parallelized by opponent, so per-opponent `elapsed_seconds` values overlap and
|
parallelized by opponent, so per-opponent `elapsed_seconds` values overlap and
|
||||||
must not be summed as wall-clock time. The slow safe-heuristic opponents
|
must not be summed as wall-clock time. The slow heuristic opponents
|
||||||
dominated the eval wall-clock.
|
dominated the eval wall-clock.
|
||||||
|
|
||||||
Representative per-opponent breakdown:
|
Representative per-opponent breakdown:
|
||||||
@@ -190,13 +190,13 @@ Representative per-opponent breakdown:
|
|||||||
| Opponent | Elapsed | Network | Postprocess | Opponent act |
|
| Opponent | Elapsed | Network | Postprocess | Opponent act |
|
||||||
| --- | ---: | ---: | ---: | ---: |
|
| --- | ---: | ---: | ---: | ---: |
|
||||||
| `random` | 0.57s | 0.18s | 0.25s | 0.06s |
|
| `random` | 0.57s | 0.18s | 0.25s | 0.06s |
|
||||||
| `passive_discard` | 0.36s | 0.13s | 0.17s | 0.00s |
|
| `discard_only` | 0.36s | 0.13s | 0.17s | 0.00s |
|
||||||
| `safe_heuristic` | 14.54s | 2.65s | 3.34s | 7.57s |
|
| `heuristic_balanced` | 14.54s | 2.65s | 3.34s | 7.57s |
|
||||||
| `safe_heuristic_loose` | 11.20s | 2.55s | 3.30s | 4.63s |
|
| `heuristic_aggressive` | 11.20s | 2.55s | 3.30s | 4.63s |
|
||||||
| `safe_heuristic_strict` | 15.94s | 2.51s | 3.14s | 9.22s |
|
| `heuristic_cautious` | 15.94s | 2.51s | 3.14s | 9.22s |
|
||||||
| `noisy_safe` | 1.68s | 0.40s | 0.58s | 0.57s |
|
| `heuristic_noisy` | 1.68s | 0.40s | 0.58s | 0.57s |
|
||||||
|
|
||||||
The important read is that safe-heuristic evaluation is not primarily GPU
|
The important read is that heuristic evaluation is not primarily GPU
|
||||||
network forward time. `opponent_act_seconds` and policy post-processing are
|
network forward time. `opponent_act_seconds` and policy post-processing are
|
||||||
larger than `policy_network_seconds` for the slowest opponents.
|
larger than `policy_network_seconds` for the slowest opponents.
|
||||||
|
|
||||||
@@ -236,7 +236,7 @@ The practical eval tuning levers are:
|
|||||||
this would require a feature change.
|
this would require a feature change.
|
||||||
|
|
||||||
4. Reduce frequent opponents.
|
4. Reduce frequent opponents.
|
||||||
The safe-heuristic opponents dominate wall-clock in the inspected run. For
|
The heuristic opponents dominate wall-clock in the inspected run. For
|
||||||
frequent checks, evaluate against one or two representative opponents and run
|
frequent checks, evaluate against one or two representative opponents and run
|
||||||
the full suite less often.
|
the full suite less often.
|
||||||
|
|
||||||
@@ -278,7 +278,7 @@ already implemented. It would replace or wrap the strategy-network forward pass
|
|||||||
with a precompiled inference engine. Its maximum impact is bounded by
|
with a precompiled inference engine. Its maximum impact is bounded by
|
||||||
`policy_network_seconds`, not by total eval time.
|
`policy_network_seconds`, not by total eval time.
|
||||||
|
|
||||||
In the inspected eval row, the slow safe-heuristic opponents spent about
|
In the inspected eval row, the slow heuristic opponents spent about
|
||||||
2.5-2.6s in policy-network forward but 4.6-9.2s in opponent action selection and
|
2.5-2.6s in policy-network forward but 4.6-9.2s in opponent action selection and
|
||||||
about 3.1-3.3s in policy post-processing. That means TensorRT could help eval,
|
about 3.1-3.3s in policy post-processing. That means TensorRT could help eval,
|
||||||
especially for larger `evaluation.games`, but it is not expected to collapse the
|
especially for larger `evaluation.games`, but it is not expected to collapse the
|
||||||
@@ -319,7 +319,7 @@ Based on the current metrics, the more plausible performance work is:
|
|||||||
but it requires changing traversal scheduling, not just swapping the network
|
but it requires changing traversal scheduling, not just swapping the network
|
||||||
backend.
|
backend.
|
||||||
|
|
||||||
6. For eval-heavy runs, optimize the safe-heuristic opponents and policy
|
6. For eval-heavy runs, optimize the heuristic opponents and policy
|
||||||
post-processing before assuming TensorRT is the main lever.
|
post-processing before assuming TensorRT is the main lever.
|
||||||
The inspected eval row shows those costs dominate the slowest opponents.
|
The inspected eval row shows those costs dominate the slowest opponents.
|
||||||
|
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ first-open advantage target.
|
|||||||
## Baseline symptoms
|
## Baseline symptoms
|
||||||
|
|
||||||
The 512x3 dense-eval baseline showed improving training losses, but the main
|
The 512x3 dense-eval baseline showed improving training losses, but the main
|
||||||
game-quality metrics against `safe_heuristic_strict` did not improve enough to
|
game-quality metrics against `heuristic_cautious` did not improve enough to
|
||||||
indicate a useful policy.
|
indicate a useful policy.
|
||||||
|
|
||||||
Observed pattern:
|
Observed pattern:
|
||||||
@@ -176,7 +176,7 @@ against the current policy's best non-open action from the same state.
|
|||||||
|
|
||||||
`delta_open = value(force open) - value(best non-open)`.
|
`delta_open = value(force open) - value(best non-open)`.
|
||||||
|
|
||||||
Counterfactual summary against `safe_heuristic_strict`:
|
Counterfactual summary against `heuristic_cautious`:
|
||||||
|
|
||||||
| checkpoint | bucket | candidates | delta mean | delta median | delta positive | policy prob | selected rate |
|
| checkpoint | bucket | candidates | delta mean | delta median | delta positive | policy prob | selected rate |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||||
@@ -257,7 +257,7 @@ first-open sampling scanned the full replay memory and pushed iteration time
|
|||||||
above 60 seconds. The indexed-memory version kept first-open sampling near
|
above 60 seconds. The indexed-memory version kept first-open sampling near
|
||||||
0.25 seconds per player at 4M advantage samples and completed 500 iterations.
|
0.25 seconds per player at 4M advantage samples and completed 500 iterations.
|
||||||
|
|
||||||
Final `safe_heuristic_strict` comparison:
|
Final `heuristic_cautious` comparison:
|
||||||
|
|
||||||
| Run | Iter | Score diff | Win rate | Bad open | Score/opened |
|
| Run | Iter | Score diff | Win rate | Bad open | Score/opened |
|
||||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||||
@@ -331,15 +331,15 @@ Result (2026-05-10):
|
|||||||
- W&B: synced online to group `first-open-prior-v1` (run `teuh915r`)
|
- W&B: synced online to group `first-open-prior-v1` (run `teuh915r`)
|
||||||
- Commit: see HEAD at run start
|
- Commit: see HEAD at run start
|
||||||
|
|
||||||
`safe_heuristic_strict` at iter 200:
|
`heuristic_cautious` at iter 200:
|
||||||
|
|
||||||
| Run | Score diff | Win rate | Bad open | Score/opened |
|
| Run | Score diff | Win rate | Bad open | Score/opened |
|
||||||
| --- | ---: | ---: | ---: | ---: |
|
| --- | ---: | ---: | ---: | ---: |
|
||||||
| baseline `confirm-eps-005-zero` (iter 200) | -40.01 | 0.12 | 0.893 | -6.25 |
|
| baseline `confirm-eps-005-zero` (iter 200) | -40.01 | 0.12 | 0.893 | -6.25 |
|
||||||
| **A1 prior α=5.0 (iter 200)** | **-67.54** | **0.02** | **0.796** | **-8.85** |
|
| **A1 prior α=5.0 (iter 200)** | **-67.54** | **0.02** | **0.796** | **-8.85** |
|
||||||
|
|
||||||
Other opponents at iter 200: random +32.51 / 0.86, noisy_safe -71.52 / 0.07,
|
Other opponents at iter 200: random +32.51 / 0.86, heuristic_noisy -71.52 / 0.07,
|
||||||
safe_heuristic -81.81 / 0.02, safe_heuristic_loose -81.24 / 0.05.
|
heuristic_balanced -81.81 / 0.02, heuristic_aggressive -81.24 / 0.05.
|
||||||
|
|
||||||
Conclusion: **mixed result, net regression.** The prior did shift behavior
|
Conclusion: **mixed result, net regression.** The prior did shift behavior
|
||||||
in the intended direction on one axis — `bad_open_rate` dropped from 0.893
|
in the intended direction on one axis — `bad_open_rate` dropped from 0.893
|
||||||
@@ -382,7 +382,7 @@ post-action play (selection bias)?
|
|||||||
|
|
||||||
Method: re-ran `analyze_first_open_counterfactual.py` on the
|
Method: re-ran `analyze_first_open_counterfactual.py` on the
|
||||||
`confirm-eps-005-zero-512x3-det-500` baseline checkpoints (iter 200, iter
|
`confirm-eps-005-zero-512x3-det-500` baseline checkpoints (iter 200, iter
|
||||||
500), but with `--post-policy safe_heuristic_strict`. The opponent and
|
500), but with `--post-policy heuristic_cautious`. The opponent and
|
||||||
state-collection policy stayed the same; only the policy_player's actions
|
state-collection policy stayed the same; only the policy_player's actions
|
||||||
*after* the forced first action used the strong fixed bot.
|
*after* the forced first action used the strong fixed bot.
|
||||||
|
|
||||||
@@ -434,7 +434,7 @@ Implications:
|
|||||||
|
|
||||||
Candidate next directions (decision pending):
|
Candidate next directions (decision pending):
|
||||||
|
|
||||||
- (E1) Train with `cutoff_rollout_policy=safe_heuristic` instead of `random`.
|
- (E1) Train with `cutoff_rollout_policy=heuristic_balanced` instead of `random`.
|
||||||
Already a config option; gives leaf nodes stronger value estimates
|
Already a config option; gives leaf nodes stronger value estimates
|
||||||
during traversal. Trades some pure-self-play purity for a stronger
|
during traversal. Trades some pure-self-play purity for a stronger
|
||||||
bootstrap signal. Cheap to test.
|
bootstrap signal. Cheap to test.
|
||||||
@@ -510,7 +510,7 @@ This combined with D1 means:
|
|||||||
|
|
||||||
Updated next-step priorities:
|
Updated next-step priorities:
|
||||||
|
|
||||||
- **(E1) `cutoff_rollout_policy=safe_heuristic` training ablation.**
|
- **(E1) `cutoff_rollout_policy=heuristic_balanced` training ablation.**
|
||||||
Strongest single lever: gives traversal leaves stronger value estimates
|
Strongest single lever: gives traversal leaves stronger value estimates
|
||||||
during training, which should ripple back to "open + follow up" signal.
|
during training, which should ripple back to "open + follow up" signal.
|
||||||
Pure-self-play purity dented, but only at cutoff leaves.
|
Pure-self-play purity dented, but only at cutoff leaves.
|
||||||
@@ -635,7 +635,7 @@ Fix:
|
|||||||
|
|
||||||
Also bumped `traversal.outcome_sampling_epsilon` in `default.yaml` from
|
Also bumped `traversal.outcome_sampling_epsilon` in `default.yaml` from
|
||||||
0.2 to 0.05. The 200-iteration sweep (section 1) showed 0.05 produced the
|
0.2 to 0.05. The 200-iteration sweep (section 1) showed 0.05 produced the
|
||||||
best short-run safe_heuristic_strict score diff (-40.01 vs -57.87 for
|
best short-run heuristic_cautious score diff (-40.01 vs -57.87 for
|
||||||
0.20). All recent experimental runs already used 0.05; the default now
|
0.20). All recent experimental runs already used 0.05; the default now
|
||||||
matches actual experimental practice.
|
matches actual experimental practice.
|
||||||
|
|
||||||
@@ -643,7 +643,71 @@ These changes do not target the diagnosed selection-bias bottleneck. They
|
|||||||
align config intent with actual scheduler behaviour and make the default
|
align config intent with actual scheduler behaviour and make the default
|
||||||
config reproduce known-best knob settings out of the box.
|
config reproduce known-best knob settings out of the box.
|
||||||
|
|
||||||
### 9. Short open-selectivity ablation
|
### 9. Naming, plot curation, and tiered eval cadence (2026-05-10)
|
||||||
|
|
||||||
|
Hygiene changes — none target the diagnosed selection-bias bottleneck,
|
||||||
|
but they make the codebase honestly reflect the pure-self-play stance
|
||||||
|
and reduce dashboard noise.
|
||||||
|
|
||||||
|
Bot family rename (drop the unhelpful `safe_` prefix; suffixes now
|
||||||
|
describe behaviour):
|
||||||
|
|
||||||
|
| Old | New |
|
||||||
|
| --- | --- |
|
||||||
|
| `safe_heuristic_loose` | `heuristic_aggressive` |
|
||||||
|
| `safe_heuristic` | `heuristic_balanced` |
|
||||||
|
| `safe_heuristic_strict` | `heuristic_cautious` |
|
||||||
|
| `noisy_safe` | `heuristic_noisy` |
|
||||||
|
| `passive_discard` | `discard_only` |
|
||||||
|
|
||||||
|
Class renames in `bots/`: `SafeHeuristicBot` → `HeuristicBot`,
|
||||||
|
`SafeHeuristicParams` → `HeuristicParams`, `PassiveDiscardBot` →
|
||||||
|
`DiscardOnlyBot`, plus the loose/strict parameter constants. Backwards
|
||||||
|
compatibility was dropped intentionally — no aliases. Active configs,
|
||||||
|
docs, scripts, tests updated; archive files (read-only by policy)
|
||||||
|
left intact and may still reference old names.
|
||||||
|
|
||||||
|
Analyze plot curation (`deep_cfr/analyze.py`):
|
||||||
|
|
||||||
|
- New `analysis_00_core.png` dashboard with 10 heuristic-free metrics
|
||||||
|
(loss/{advantage, strategy}; vs `heuristic_cautious`:
|
||||||
|
`avg_score_diff0`, `win_rate0`, `avg_opened_colors`,
|
||||||
|
`positive_expedition_rate`, `bonus_expedition_rate`,
|
||||||
|
`score_per_opened_color`, `policy_entropy`; vs `random`: `win_rate0`).
|
||||||
|
- Removed `analysis_05_open_quality.png` (bad/weak/good open rates,
|
||||||
|
recoverable score) and `analysis_07_calibration.png` (calibration gap,
|
||||||
|
recoverable mean) — both relied on the heuristic `recoverable_score`
|
||||||
|
classifier we already dropped from inputs.
|
||||||
|
- Removed `SELECTIVITY_PLOTS` and `plot_selectivity` (heuristic-laden).
|
||||||
|
- `SUMMARY_EVAL_METRICS` no longer includes `bad_open_rate` or
|
||||||
|
`calibration_gap`.
|
||||||
|
|
||||||
|
`PlotSpec` gained an optional `opponents` allowlist so the new core
|
||||||
|
section can pin a specific opponent per panel without restructuring the
|
||||||
|
existing `plot_section` plumbing.
|
||||||
|
|
||||||
|
Tiered evaluation cadence (`EvaluationConfig`):
|
||||||
|
|
||||||
|
- Added `extended_opponents: tuple[str, ...]` and `extended_eval_every:
|
||||||
|
int = 0`.
|
||||||
|
- Method `opponents_for_iteration(iteration)` returns the core list at
|
||||||
|
every `eval_every`, and appends `extended_opponents` (de-duplicated)
|
||||||
|
when `iteration` is also a multiple of `extended_eval_every`.
|
||||||
|
- `default.yaml` now uses 3 core opponents
|
||||||
|
(`random`, `discard_only`, `heuristic_cautious`) every 5 iterations
|
||||||
|
and 3 extended opponents (`heuristic_balanced`, `heuristic_aggressive`,
|
||||||
|
`heuristic_noisy`) every 50 iterations.
|
||||||
|
- `random` is the floor sanity. `discard_only` is the zero-pit
|
||||||
|
detector / absolute-score reference (its score is always 0, so
|
||||||
|
`eval/discard_only/avg_score_diff0` directly equals our model's
|
||||||
|
raw average score). `heuristic_cautious` is the ceiling and the
|
||||||
|
archive-comparable benchmark used in sections 1–6.
|
||||||
|
|
||||||
|
Net effect on ongoing eval cost: ~50% reduction (3 opponents × every
|
||||||
|
5 iter, plus 6 opponents × every 50 iter, vs the prior 6 opponents
|
||||||
|
× every 5).
|
||||||
|
|
||||||
|
### 10. Short open-selectivity ablation
|
||||||
|
|
||||||
Run a 200-300 iteration ablation only after the target audit identifies a
|
Run a 200-300 iteration ablation only after the target audit identifies a
|
||||||
specific change. Candidate changes include:
|
specific change. Candidate changes include:
|
||||||
@@ -656,10 +720,10 @@ specific change. Candidate changes include:
|
|||||||
|
|
||||||
Primary metrics:
|
Primary metrics:
|
||||||
|
|
||||||
- `eval/safe_heuristic_strict/avg_score_diff0`
|
- `eval/heuristic_cautious/avg_score_diff0`
|
||||||
- `eval/safe_heuristic_strict/win_rate0`
|
- `eval/heuristic_cautious/win_rate0`
|
||||||
- `eval/safe_heuristic_strict/bad_open_rate`
|
- `eval/heuristic_cautious/bad_open_rate`
|
||||||
- `eval/safe_heuristic_strict/score_per_opened_color`
|
- `eval/heuristic_cautious/score_per_opened_color`
|
||||||
|
|
||||||
Do not promote to 500+ iterations unless bad-open rate and score/opened color
|
Do not promote to 500+ iterations unless bad-open rate and score/opened color
|
||||||
both improve without degrading score diff.
|
both improve without degrading score diff.
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ The experiment must produce either a recommended new `network` config or a docum
|
|||||||
|
|
||||||
## Success criteria
|
## Success criteria
|
||||||
|
|
||||||
1. At least one tested config produces win-rate trajectories vs `safe_heuristic_strict` that are **clearly outside seed noise** compared to the current baseline at iteration 200 — OR a clear documented null result (no size in the tested range improves the curve).
|
1. At least one tested config produces win-rate trajectories vs `heuristic_cautious` that are **clearly outside seed noise** compared to the current baseline at iteration 200 — OR a clear documented null result (no size in the tested range improves the curve).
|
||||||
2. `iteration_seconds`, `traversal_seconds`, `advantage_train_seconds`, `strategy_train_seconds`, and `policy_network_seconds` (eval) are captured for each tested size and written to `docs/performance.md`.
|
2. `iteration_seconds`, `traversal_seconds`, `advantage_train_seconds`, `strategy_train_seconds`, and `policy_network_seconds` (eval) are captured for each tested size and written to `docs/performance.md`.
|
||||||
3. A recommended `network` config emerges from the data, OR the experiment documents why the current size should be kept, with specific rationale.
|
3. A recommended `network` config emerges from the data, OR the experiment documents why the current size should be kept, with specific rationale.
|
||||||
|
|
||||||
@@ -72,7 +72,7 @@ configs/deep_cfr/model-size-1536x8.yaml
|
|||||||
- `traversal_seconds` — traversal phase.
|
- `traversal_seconds` — traversal phase.
|
||||||
- `advantage_train_seconds` — advantage network optimization.
|
- `advantage_train_seconds` — advantage network optimization.
|
||||||
- `strategy_train_seconds` — strategy network optimization.
|
- `strategy_train_seconds` — strategy network optimization.
|
||||||
- At eval iterations {50, 100, 150, 200}: `eval/<opponent>/win_rate` for all opponents, with special attention to `safe_heuristic_strict`.
|
- At eval iterations {50, 100, 150, 200}: `eval/<opponent>/win_rate` for all opponents, with special attention to `heuristic_cautious`.
|
||||||
- At eval iterations: `eval/<opponent>/policy_network_seconds` — needed for the AMP/TRT prerequisite check.
|
- At eval iterations: `eval/<opponent>/policy_network_seconds` — needed for the AMP/TRT prerequisite check.
|
||||||
- **Memory monitoring:** watch GPU VRAM during the 1024x6 and 1536x8 runs. If a run OOMs or VRAM > 20 GB, reduce `optimization.advantage_batch_size` and `optimization.strategy_batch_size` by half (1024 → 512) and note the change in the results table. Do not adjust traversal settings.
|
- **Memory monitoring:** watch GPU VRAM during the 1024x6 and 1536x8 runs. If a run OOMs or VRAM > 20 GB, reduce `optimization.advantage_batch_size` and `optimization.strategy_batch_size` by half (1024 → 512) and note the change in the results table. Do not adjust traversal settings.
|
||||||
|
|
||||||
@@ -259,8 +259,8 @@ if non_eval:
|
|||||||
eval_rows = {r['iteration']: r for r in rows if r.get('evaluation_seconds')}
|
eval_rows = {r['iteration']: r for r in rows if r.get('evaluation_seconds')}
|
||||||
for it in [50, 100, 150, 200]:
|
for it in [50, 100, 150, 200]:
|
||||||
if it in eval_rows:
|
if it in eval_rows:
|
||||||
wr = eval_rows[it].get('eval/safe_heuristic_strict/win_rate', 'n/a')
|
wr = eval_rows[it].get('eval/heuristic_cautious/win_rate', 'n/a')
|
||||||
print(f' iter={it} safe_heuristic_strict win_rate={wr}')
|
print(f' iter={it} heuristic_cautious win_rate={wr}')
|
||||||
"
|
"
|
||||||
done
|
done
|
||||||
```
|
```
|
||||||
@@ -269,7 +269,7 @@ done
|
|||||||
|
|
||||||
After the grid completes, append a date-stamped experiment subsection to `docs/performance.md` under the "Experiments" heading. The subsection must include:
|
After the grid completes, append a date-stamped experiment subsection to `docs/performance.md` under the "Experiments" heading. The subsection must include:
|
||||||
|
|
||||||
- A results table with `iteration_seconds` mean (non-eval) and win-rate vs `safe_heuristic_strict` at {50, 100, 150, 200} for each config.
|
- A results table with `iteration_seconds` mean (non-eval) and win-rate vs `heuristic_cautious` at {50, 100, 150, 200} for each config.
|
||||||
- A `policy_network_seconds` column from eval rows — this is the key data for the AMP/compile/TRT prerequisite check.
|
- A `policy_network_seconds` column from eval rows — this is the key data for the AMP/compile/TRT prerequisite check.
|
||||||
- The recommendation that follows from the decision tree below.
|
- The recommendation that follows from the decision tree below.
|
||||||
|
|
||||||
@@ -302,7 +302,7 @@ Apply this logic after the grid completes:
|
|||||||
|
|
||||||
### Branch A — a size unlocks the curve AND iter time is acceptable
|
### Branch A — a size unlocks the curve AND iter time is acceptable
|
||||||
|
|
||||||
**Condition:** at least one config at or above 768x4 shows win-rate trajectories vs `safe_heuristic_strict` that are clearly outside seed noise vs 512x3 baseline at iteration 200, AND `iteration_seconds` at that size is ≤ 3× the baseline (i.e., ≤ ~54s/iter).
|
**Condition:** at least one config at or above 768x4 shows win-rate trajectories vs `heuristic_cautious` that are clearly outside seed noise vs 512x3 baseline at iteration 200, AND `iteration_seconds` at that size is ≤ 3× the baseline (i.e., ≤ ~54s/iter).
|
||||||
|
|
||||||
**Action:**
|
**Action:**
|
||||||
1. Recommend that config as the new `network` default.
|
1. Recommend that config as the new `network` default.
|
||||||
|
|||||||
@@ -152,7 +152,7 @@ Cython production rewrite. It intentionally uses per-context RNG so interleaved
|
|||||||
execution order does not change the random stream for another context. That
|
execution order does not change the random stream for another context. That
|
||||||
lets the prototype assert value/stat/sample parity against a recursive prototype
|
lets the prototype assert value/stat/sample parity against a recursive prototype
|
||||||
while measuring realized batch size. Production Cython parity is a later Phase 2
|
while measuring realized batch size. Production Cython parity is a later Phase 2
|
||||||
gate because the real path also has safe-heuristic opponents, average-strategy
|
gate because the real path also has heuristic-balanced opponents, average-strategy
|
||||||
opponents, self-play league snapshots, deck-draw chance sampling, external
|
opponents, self-play league snapshots, deck-draw chance sampling, external
|
||||||
sampling, and cutoff rollouts.
|
sampling, and cutoff rollouts.
|
||||||
|
|
||||||
@@ -312,7 +312,7 @@ Required feature expansion:
|
|||||||
|
|
||||||
- Support `opponent_policy: average_strategy`, matching the default config's
|
- Support `opponent_policy: average_strategy`, matching the default config's
|
||||||
opponent branch.
|
opponent branch.
|
||||||
- Keep unsupported branches guarded (`self_play_league`, `safe_heuristic`,
|
- Keep unsupported branches guarded (`self_play_league`, `heuristic_balanced`,
|
||||||
random rollout cutoffs, external sampling).
|
random rollout cutoffs, external sampling).
|
||||||
- Add parity tests for the average-strategy fixed-opponent branch.
|
- Add parity tests for the average-strategy fixed-opponent branch.
|
||||||
- Verify a default-policy interleaved run starts and emits batch metrics.
|
- Verify a default-policy interleaved run starts and emits batch metrics.
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ Re-enable `torch.compile` on the Deep CFR trainer's networks at a model size whe
|
|||||||
|
|
||||||
1. **Model-size precondition met.** The active `default.yaml` (or the targeted variant) has `network.hidden_size ≥ 1024` *or* `network.num_layers ≥ 6`, *or* an architecture (e.g. `color_shared` with non-trivial `color_attention_layers`) whose per-call forward time exceeds ~150 μs at the trainer's training batch size on the target GPU. If neither condition holds, this plan is **not merged**; the branch is parked.
|
1. **Model-size precondition met.** The active `default.yaml` (or the targeted variant) has `network.hidden_size ≥ 1024` *or* `network.num_layers ≥ 6`, *or* an architecture (e.g. `color_shared` with non-trivial `color_attention_layers`) whose per-call forward time exceeds ~150 μs at the trainer's training batch size on the target GPU. If neither condition holds, this plan is **not merged**; the branch is parked.
|
||||||
2. **Iter-time improvement.** With `compile.trainer.enabled: true` on the chosen larger model and the same seed, the 1000-iter projection improves by at least **5%** vs the no-compile baseline on the same machine (measured on `home`). Eval and checkpointing should be disabled for the bench window, matching the protocol used in the 2026-05-07 experiment.
|
2. **Iter-time improvement.** With `compile.trainer.enabled: true` on the chosen larger model and the same seed, the 1000-iter projection improves by at least **5%** vs the no-compile baseline on the same machine (measured on `home`). Eval and checkpointing should be disabled for the bench window, matching the protocol used in the 2026-05-07 experiment.
|
||||||
3. **No learning-curve drift.** Over at least 100 iterations with `compile.trainer.enabled: true` vs `false` (same seed, same config), the eval win-rate trajectories against `random` and `safe_heuristic` are within seed noise. If trajectories visibly diverge, the plan does not ship even if iter time improves.
|
3. **No learning-curve drift.** Over at least 100 iterations with `compile.trainer.enabled: true` vs `false` (same seed, same config), the eval win-rate trajectories against `random` and `heuristic_balanced` are within seed noise. If trajectories visibly diverge, the plan does not ship even if iter time improves.
|
||||||
4. **No checkpoint-format break.** Checkpoints saved with compile enabled must load cleanly when compile is disabled, and vice versa. (Handled via `_clean_state_dict()`; see Risks.)
|
4. **No checkpoint-format break.** Checkpoints saved with compile enabled must load cleanly when compile is disabled, and vice versa. (Handled via `_clean_state_dict()`; see Risks.)
|
||||||
5. **No multiprocessing-worker break.** Whether `inference_backend` is `local` or `server`, traversal workers must continue to receive uncompiled `state_dict`s without `_orig_mod.` prefixes.
|
5. **No multiprocessing-worker break.** Whether `inference_backend` is `local` or `server`, traversal workers must continue to receive uncompiled `state_dict`s without `_orig_mod.` prefixes.
|
||||||
6. **(Secondary) Inference-server forward.** If step B below is taken, the server's `policy_network_seconds` decreases by at least 20% at the chosen model size, with no traversal-path correctness regression. If step B does not produce a measurable win, it is left disabled and the plan still ships with step A only.
|
6. **(Secondary) Inference-server forward.** If step B below is taken, the server's `policy_network_seconds` decreases by at least 20% at the chosen model size, with no traversal-path correctness regression. If step B does not produce a measurable win, it is left disabled and the plan still ships with step A only.
|
||||||
|
|||||||
@@ -157,9 +157,9 @@ Python-object touch가 깔려 있다 (per-node, per-iteration):
|
|||||||
`self.strategy_samples.append(...)` (file:903, 937, 969). list의
|
`self.strategy_samples.append(...)` (file:903, 937, 969). list의
|
||||||
PyObject reference 갱신은 free-threaded Python에서도 atomic refcount
|
PyObject reference 갱신은 free-threaded Python에서도 atomic refcount
|
||||||
비용을 추가로 부담한다.
|
비용을 추가로 부담한다.
|
||||||
6. **`SafeHeuristicBot.act(state)`** — `_fixed_opponent_action`
|
6. **`HeuristicBot.act(state)`** — `_fixed_opponent_action`
|
||||||
(file:633, 652), `_rollout_value` (file:841). Python class
|
(file:633, 652), `_rollout_value` (file:841). Python class
|
||||||
메서드 호출. `safe_heuristic` 옵션 사용 시만 핫.
|
메서드 호출. `heuristic_balanced` 옵션 사용 시만 핫.
|
||||||
7. **`league_advantage_networks` indexing** — `_self_play_snapshot_
|
7. **`league_advantage_networks` indexing** — `_self_play_snapshot_
|
||||||
networks` (file:802), `[-recent_count:]`, `[:max(0, ...)]` slicing
|
networks` (file:802), `[-recent_count:]`, `[:max(0, ...)]` slicing
|
||||||
= Python list slicing.
|
= Python list slicing.
|
||||||
@@ -267,15 +267,15 @@ candidates = self.league_advantage_networks[:max(0, len(self.league_advantage_ne
|
|||||||
- 빈도: traversal 진입 시 한 번 (`traverse`에서 미리 픽), 재귀 안에서는
|
- 빈도: traversal 진입 시 한 번 (`traverse`에서 미리 픽), 재귀 안에서는
|
||||||
`active_self_play_networks`만 본다. 따라서 cold path. 변환 불필요.
|
`active_self_play_networks`만 본다. 따라서 cold path. 변환 불필요.
|
||||||
|
|
||||||
### B6. `SafeHeuristicBot.act(state)` — Python bot
|
### B6. `HeuristicBot.act(state)` — Python bot
|
||||||
|
|
||||||
```python
|
```python
|
||||||
# traversal.pyx:633, 652, 841
|
# traversal.pyx:633, 652, 841
|
||||||
return int(self.safe_heuristic_opponent_bot.act(state))
|
return int(self.heuristic_opponent_bot.act(state))
|
||||||
```
|
```
|
||||||
|
|
||||||
- 빈도: `opponent_policy=safe_heuristic` 또는 `cutoff_rollout_policy=
|
- 빈도: `opponent_policy=heuristic_balanced` 또는 `cutoff_rollout_policy=
|
||||||
safe_heuristic`일 때만. 현 default는 self_play_league + score_diff
|
heuristic_balanced`일 때만. 현 default는 self_play_league + score_diff
|
||||||
cutoff (per memory의 opponent_policy_network_divergence note + AGENTS).
|
cutoff (per memory의 opponent_policy_network_divergence note + AGENTS).
|
||||||
- 변환 난이도: **Medium-High** (Python class 전체를 cython화). 현 default
|
- 변환 난이도: **Medium-High** (Python class 전체를 cython화). 현 default
|
||||||
config에서는 핫 아님 — 시도하지 않는 게 합리.
|
config에서는 핫 아님 — 시도하지 않는 게 합리.
|
||||||
|
|||||||
@@ -48,7 +48,7 @@ src/coolrl_lost_cities/games/classic/deep_cfr/
|
|||||||
traversal_stats.py — structured traversal diagnostic metrics
|
traversal_stats.py — structured traversal diagnostic metrics
|
||||||
|
|
||||||
# Auxiliary training modes
|
# Auxiliary training modes
|
||||||
imitation.py — safe-heuristic imitation pretraining
|
imitation.py — heuristic imitation pretraining
|
||||||
policy_gradient.py — policy-gradient fine-tuning
|
policy_gradient.py — policy-gradient fine-tuning
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -59,23 +59,23 @@ Evaluation performance peaks early and then degrades.
|
|||||||
| Opponent | First 20 evals `win_rate0` | Last 20 evals `win_rate0` | First 20 evals score diff | Last 20 evals score diff |
|
| Opponent | First 20 evals `win_rate0` | Last 20 evals `win_rate0` | First 20 evals score diff | Last 20 evals score diff |
|
||||||
| --- | ---: | ---: | ---: | ---: |
|
| --- | ---: | ---: | ---: | ---: |
|
||||||
| random | 0.808 | 0.665 | +35.1 | +12.4 |
|
| random | 0.808 | 0.665 | +35.1 | +12.4 |
|
||||||
| passive_discard | 0.026 | 0.012 | -30.8 | -43.0 |
|
| discard_only | 0.026 | 0.012 | -30.8 | -43.0 |
|
||||||
| safe_heuristic | 0.070 | 0.011 | -68.7 | -94.2 |
|
| heuristic_balanced | 0.070 | 0.011 | -68.7 | -94.2 |
|
||||||
| safe_heuristic_loose | 0.078 | 0.014 | -69.3 | -95.0 |
|
| heuristic_aggressive | 0.078 | 0.014 | -69.3 | -95.0 |
|
||||||
| safe_heuristic_strict | 0.072 | 0.011 | -57.6 | -85.0 |
|
| heuristic_cautious | 0.072 | 0.011 | -57.6 | -85.0 |
|
||||||
| noisy_safe | 0.120 | 0.033 | -49.4 | -78.8 |
|
| heuristic_noisy | 0.120 | 0.033 | -49.4 | -78.8 |
|
||||||
|
|
||||||
The best strict-heuristic point appears around iteration 70:
|
The best strict-heuristic point appears around iteration 70:
|
||||||
|
|
||||||
- `eval/safe_heuristic_strict/win_rate0=0.15`
|
- `eval/heuristic_cautious/win_rate0=0.15`
|
||||||
- `eval/safe_heuristic_strict/avg_score_diff0=-41.09`
|
- `eval/heuristic_cautious/avg_score_diff0=-41.09`
|
||||||
|
|
||||||
The final point at iteration 2000 is worse:
|
The final point at iteration 2000 is worse:
|
||||||
|
|
||||||
- `eval/safe_heuristic_strict/win_rate0=0.02`
|
- `eval/heuristic_cautious/win_rate0=0.02`
|
||||||
- `eval/safe_heuristic_strict/avg_score_diff0=-81.64`
|
- `eval/heuristic_cautious/avg_score_diff0=-81.64`
|
||||||
|
|
||||||
Selectivity does not meaningfully emerge. Against `safe_heuristic_strict`:
|
Selectivity does not meaningfully emerge. Against `heuristic_cautious`:
|
||||||
|
|
||||||
| Metric | First 20 evals | Last 20 evals | Interpretation |
|
| Metric | First 20 evals | Last 20 evals | Interpretation |
|
||||||
| --- | ---: | ---: | --- |
|
| --- | ---: | ---: | --- |
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ The primary bottleneck in CUDA-based evaluation is the overhead of launching sma
|
|||||||
|
|
||||||
A critical refinement in the batched implementation was the handling of policy entropy. Initial versions that calculated entropy per-row on the CPU incurred significant synchronization penalties because each row required a GPU-to-CPU transfer. Moving the entropy calculation into the Torch post-processing pipeline—specifically calculating it directly on the `probs_tensor` (line 220)—ensures that the computation remains on the device and only the final results are transferred back to the host in bulk.
|
A critical refinement in the batched implementation was the handling of policy entropy. Initial versions that calculated entropy per-row on the CPU incurred significant synchronization penalties because each row required a GPU-to-CPU transfer. Moving the entropy calculation into the Torch post-processing pipeline—specifically calculating it directly on the `probs_tensor` (line 220)—ensures that the computation remains on the device and only the final results are transferred back to the host in bulk.
|
||||||
|
|
||||||
Once network inference is batched, the remaining bottleneck often shifts to the CPU-bound logic of heuristic opponents (e.g., `safe_heuristic_strict`). Parallelizing the evaluation across multiple workers allows the trainer to evaluate against multiple opponents simultaneously. For a single iteration profile, using 4 parallel workers reduced wall-clock evaluation time from 14.8 seconds to 6.4 seconds, achieving a ~2.3x speedup.
|
Once network inference is batched, the remaining bottleneck often shifts to the CPU-bound logic of heuristic opponents (e.g., `heuristic_cautious`). Parallelizing the evaluation across multiple workers allows the trainer to evaluate against multiple opponents simultaneously. For a single iteration profile, using 4 parallel workers reduced wall-clock evaluation time from 14.8 seconds to 6.4 seconds, achieving a ~2.3x speedup.
|
||||||
|
|
||||||
## Practical implication
|
## Practical implication
|
||||||
|
|
||||||
|
|||||||
@@ -37,7 +37,7 @@ On CPU, the `network / turn` cost is approximately **0.074 ms**. On CUDA, this r
|
|||||||
|
|
||||||
### Secondary Bottlenecks
|
### Secondary Bottlenecks
|
||||||
- **Post-processing:** Moving tensors back to CPU (`.cpu().numpy()`) and calculating entropy adds measurable overhead on CUDA that is largely absent on CPU.
|
- **Post-processing:** Moving tensors back to CPU (`.cpu().numpy()`) and calculating entropy adds measurable overhead on CUDA that is largely absent on CPU.
|
||||||
- **Opponent Logic:** Heuristic opponents (e.g., `safe_heuristic`) contribute significant `opponent_act_seconds` (up to 3.5s per eval iteration). Since this logic is pure Python/Cython and runs on the CPU, it does not benefit from GPU acceleration, further diluting any potential CUDA wins.
|
- **Opponent Logic:** Heuristic opponents (e.g., `heuristic_balanced`) contribute significant `opponent_act_seconds` (up to 3.5s per eval iteration). Since this logic is pure Python/Cython and runs on the CPU, it does not benefit from GPU acceleration, further diluting any potential CUDA wins.
|
||||||
|
|
||||||
## Practical Implications
|
## Practical Implications
|
||||||
|
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ Two temporary runs used:
|
|||||||
- `run.max_iterations=3`
|
- `run.max_iterations=3`
|
||||||
- `evaluation.eval_every=1`
|
- `evaluation.eval_every=1`
|
||||||
- `evaluation.games=20`
|
- `evaluation.games=20`
|
||||||
- `evaluation.opponents=[random,safe_heuristic_strict]`
|
- `evaluation.opponents=[random,heuristic_cautious]`
|
||||||
- W&B disabled
|
- W&B disabled
|
||||||
|
|
||||||
Runs:
|
Runs:
|
||||||
@@ -55,7 +55,7 @@ Runs:
|
|||||||
|
|
||||||
Core metrics matched exactly:
|
Core metrics matched exactly:
|
||||||
|
|
||||||
| Iteration | `traversal/nodes` | `memory/advantage` | `loss/advantage` | `eval/random/win_rate0` | `eval/safe_heuristic_strict/win_rate0` |
|
| Iteration | `traversal/nodes` | `memory/advantage` | `loss/advantage` | `eval/random/win_rate0` | `eval/heuristic_cautious/win_rate0` |
|
||||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||||
| 1 | 170822 | 85096 | 803.9475702643394 | 0.75 | 0.10 |
|
| 1 | 170822 | 85096 | 803.9475702643394 | 0.75 | 0.10 |
|
||||||
| 2 | 206225 | 187987 | 821.3355012834072 | 0.50 | 0.05 |
|
| 2 | 206225 | 187987 | 821.3355012834072 | 0.50 | 0.05 |
|
||||||
|
|||||||
@@ -56,7 +56,7 @@ registered classic bots (`evaluate.py`), training CLI and evaluation CLI
|
|||||||
(`cli.py`), traversal benchmark CLI (`benchmark.py`), `metrics.jsonl` /
|
(`cli.py`), traversal benchmark CLI (`benchmark.py`), `metrics.jsonl` /
|
||||||
`runtime_progress.json` / `train.log` run artifacts, self-play league with
|
`runtime_progress.json` / `train.log` run artifacts, self-play league with
|
||||||
snapshot pool and weighted current/recent/older/anchor bucket sampling, safe-
|
snapshot pool and weighted current/recent/older/anchor bucket sampling, safe-
|
||||||
heuristic anchor opponent, safe-heuristic imitation pretraining (`imitation.py`),
|
heuristic anchor opponent, heuristic imitation pretraining (`imitation.py`),
|
||||||
and policy-gradient fine-tuning (`policy_gradient.py`).
|
and policy-gradient fine-tuning (`policy_gradient.py`).
|
||||||
|
|
||||||
As of `ad0be89`, the package also includes `inference_server.py`,
|
As of `ad0be89`, the package also includes `inference_server.py`,
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ Decision criteria for going forward:
|
|||||||
|
|
||||||
### 2026-05-07 — Safe heuristic single-thread parity (criterion 1)
|
### 2026-05-07 — Safe heuristic single-thread parity (criterion 1)
|
||||||
|
|
||||||
Path: `experiments/julia_safe_heuristic/`.
|
Path: `experiments/julia_heuristic/`.
|
||||||
|
|
||||||
1,838 snapshots in 157.471 ms median (~85.6 μs/call). Action-sequence
|
1,838 snapshots in 157.471 ms median (~85.6 μs/call). Action-sequence
|
||||||
parity vs Python. Same order of magnitude as the Cython port of the
|
parity vs Python. Same order of magnitude as the Cython port of the
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ Deep CFR은 최종적으로 average strategy가 수렴 대상. current advantage
|
|||||||
Visible-score predictor의 ceiling은 있을 수 있음. 다만 selectivity 자체의 ceiling은 아님. selectivity는 visible-score prediction 말고 option value, irreversible cost 회피, opponent dynamics 대응 등 다른 경로로도 emerge 가능.
|
Visible-score predictor의 ceiling은 있을 수 있음. 다만 selectivity 자체의 ceiling은 아님. selectivity는 visible-score prediction 말고 option value, irreversible cost 회피, opponent dynamics 대응 등 다른 경로로도 emerge 가능.
|
||||||
|
|
||||||
**Self-play attractor 가설**
|
**Self-play attractor 가설**
|
||||||
5-color가 stable equilibrium. 한쪽이 selectivity 시도하면 즉시 손해. safe_heuristic이 3.7에서 강하면 진짜 NE는 아닐 듯. 다만 self-play 안에서는 stable.
|
5-color가 stable equilibrium. 한쪽이 selectivity 시도하면 즉시 손해. heuristic_balanced이 3.7에서 강하면 진짜 NE는 아닐 듯. 다만 self-play 안에서는 stable.
|
||||||
|
|
||||||
**Lost Cities NE 자체가 5-color 가설**
|
**Lost Cities NE 자체가 5-color 가설**
|
||||||
이론적 가능성 0 아님. self-play가 발견한 게 진짜 NE면 transition 영원히 안 옴. 검증 가능한 형태로는 tabular oracle 또는 BR 진단.
|
이론적 가능성 0 아님. self-play가 발견한 게 진짜 NE면 transition 영원히 안 옴. 검증 가능한 형태로는 tabular oracle 또는 BR 진단.
|
||||||
|
|||||||
@@ -240,7 +240,7 @@ def main() -> None:
|
|||||||
parser.add_argument("--output", type=Path, required=True)
|
parser.add_argument("--output", type=Path, required=True)
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
|
|
||||||
opponents = args.opponent or ["safe_heuristic_strict"]
|
opponents = args.opponent or ["heuristic_cautious"]
|
||||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
rows = []
|
rows = []
|
||||||
for checkpoint in args.checkpoints:
|
for checkpoint in args.checkpoints:
|
||||||
|
|||||||
@@ -367,7 +367,7 @@ def analyze_checkpoint(
|
|||||||
def main() -> None:
|
def main() -> None:
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
parser.add_argument("checkpoints", nargs="+", type=Path)
|
parser.add_argument("checkpoints", nargs="+", type=Path)
|
||||||
parser.add_argument("--opponent", default="safe_heuristic_strict")
|
parser.add_argument("--opponent", default="heuristic_cautious")
|
||||||
parser.add_argument("--games", type=int, default=100)
|
parser.add_argument("--games", type=int, default=100)
|
||||||
parser.add_argument("--seed", type=int, default=231_000)
|
parser.add_argument("--seed", type=int, default=231_000)
|
||||||
parser.add_argument("--device", default="cuda")
|
parser.add_argument("--device", default="cuda")
|
||||||
@@ -379,7 +379,7 @@ def main() -> None:
|
|||||||
help=(
|
help=(
|
||||||
"Policy used for the policy_player during forced-action rollouts. "
|
"Policy used for the policy_player during forced-action rollouts. "
|
||||||
"'model' uses the trained advantage network; any other value is "
|
"'model' uses the trained advantage network; any other value is "
|
||||||
"treated as a bot name (e.g. 'safe_heuristic_strict')."
|
"treated as a bot name (e.g. 'heuristic_cautious')."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
parser.add_argument("--output", type=Path, required=True)
|
parser.add_argument("--output", type=Path, required=True)
|
||||||
|
|||||||
@@ -273,7 +273,7 @@ def analyze_checkpoint(
|
|||||||
def main() -> None:
|
def main() -> None:
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
parser.add_argument("checkpoints", nargs="+", type=Path)
|
parser.add_argument("checkpoints", nargs="+", type=Path)
|
||||||
parser.add_argument("--opponent", default="safe_heuristic_strict")
|
parser.add_argument("--opponent", default="heuristic_cautious")
|
||||||
parser.add_argument("--games", type=int, default=100)
|
parser.add_argument("--games", type=int, default=100)
|
||||||
parser.add_argument("--seed", type=int, default=232_000)
|
parser.add_argument("--seed", type=int, default=232_000)
|
||||||
parser.add_argument("--device", default="cpu")
|
parser.add_argument("--device", default="cpu")
|
||||||
|
|||||||
@@ -265,7 +265,7 @@ def main() -> None:
|
|||||||
parser.add_argument("--output", type=Path, required=True)
|
parser.add_argument("--output", type=Path, required=True)
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
|
|
||||||
opponents = args.opponent or ["random", "safe_heuristic_strict"]
|
opponents = args.opponent or ["random", "heuristic_cautious"]
|
||||||
args.output.parent.mkdir(parents=True, exist_ok=True)
|
args.output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
rows: list[dict] = []
|
rows: list[dict] = []
|
||||||
|
|||||||
@@ -7,22 +7,22 @@ from typing import Any
|
|||||||
|
|
||||||
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig
|
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig
|
||||||
|
|
||||||
from coolrl_lost_cities.games.classic.bots.heuristic_py import SafeHeuristicBot
|
from coolrl_lost_cities.games.classic.bots.heuristic_py import HeuristicBot
|
||||||
from coolrl_lost_cities.games.classic.bots.registry import (
|
from coolrl_lost_cities.games.classic.bots.registry import (
|
||||||
LOOSE_SAFE_HEURISTIC_PARAMS,
|
AGGRESSIVE_HEURISTIC_PARAMS,
|
||||||
STRICT_SAFE_HEURISTIC_PARAMS,
|
CAUTIOUS_HEURISTIC_PARAMS,
|
||||||
)
|
)
|
||||||
|
|
||||||
VARIANTS = {
|
VARIANTS = {
|
||||||
"default": None,
|
"default": None,
|
||||||
"loose": LOOSE_SAFE_HEURISTIC_PARAMS,
|
"loose": AGGRESSIVE_HEURISTIC_PARAMS,
|
||||||
"strict": STRICT_SAFE_HEURISTIC_PARAMS,
|
"strict": CAUTIOUS_HEURISTIC_PARAMS,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def parse_args() -> argparse.Namespace:
|
def parse_args() -> argparse.Namespace:
|
||||||
parser = argparse.ArgumentParser(
|
parser = argparse.ArgumentParser(
|
||||||
description="Export safe-heuristic bot parity snapshots for external implementations."
|
description="Export heuristic bot parity snapshots for external implementations."
|
||||||
)
|
)
|
||||||
parser.add_argument("--output", required=True, help="JSONL output path.")
|
parser.add_argument("--output", required=True, help="JSONL output path.")
|
||||||
parser.add_argument("--seeds", type=int, default=50, help="Number of seeds per config.")
|
parser.add_argument("--seeds", type=int, default=50, help="Number of seeds per config.")
|
||||||
@@ -71,7 +71,7 @@ def main() -> None:
|
|||||||
for config_name, config in _configs():
|
for config_name, config in _configs():
|
||||||
for variant_name, params in VARIANTS.items():
|
for variant_name, params in VARIANTS.items():
|
||||||
for seed in range(args.seeds):
|
for seed in range(args.seeds):
|
||||||
bot = SafeHeuristicBot(params)
|
bot = HeuristicBot(params)
|
||||||
state = GameState.new_game(config, seed=seed)
|
state = GameState.new_game(config, seed=seed)
|
||||||
for turn in range(args.max_steps):
|
for turn in range(args.max_steps):
|
||||||
if state.terminal:
|
if state.terminal:
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from ..policy import LostCitiesPolicy, PolicyInput
|
from ..policy import LostCitiesPolicy, PolicyInput
|
||||||
from .heuristic import SafeHeuristicBot
|
from .discard_only import DiscardOnlyBot
|
||||||
from .passive import PassiveDiscardBot
|
from .heuristic import HeuristicBot
|
||||||
from .random import RandomBot
|
from .random import RandomBot
|
||||||
from .registry import DEFAULT_BOT, available_bot_names, build_bot
|
from .registry import DEFAULT_BOT, available_bot_names, build_bot
|
||||||
|
|
||||||
@@ -10,9 +10,9 @@ __all__ = [
|
|||||||
"PolicyInput",
|
"PolicyInput",
|
||||||
"DEFAULT_BOT",
|
"DEFAULT_BOT",
|
||||||
"LostCitiesPolicy",
|
"LostCitiesPolicy",
|
||||||
"PassiveDiscardBot",
|
"DiscardOnlyBot",
|
||||||
"RandomBot",
|
"RandomBot",
|
||||||
"SafeHeuristicBot",
|
"HeuristicBot",
|
||||||
"available_bot_names",
|
"available_bot_names",
|
||||||
"build_bot",
|
"build_bot",
|
||||||
]
|
]
|
||||||
|
|||||||
+1
-1
@@ -6,7 +6,7 @@ from ..snapshots import Snapshot
|
|||||||
from .base import first_legal, legal_from_obs
|
from .base import first_legal, legal_from_obs
|
||||||
|
|
||||||
|
|
||||||
class PassiveDiscardBot(LostCitiesPolicy):
|
class DiscardOnlyBot(LostCitiesPolicy):
|
||||||
"""Baseline that avoids opening expeditions whenever discarding is legal."""
|
"""Baseline that avoids opening expeditions whenever discarding is legal."""
|
||||||
|
|
||||||
def act(self, obs_or_state: PolicyInput) -> int:
|
def act(self, obs_or_state: PolicyInput) -> int:
|
||||||
@@ -4,8 +4,8 @@ from .heuristic_cy import (
|
|||||||
DRAW_FROM_DECK_ACTION,
|
DRAW_FROM_DECK_ACTION,
|
||||||
PLAY_OR_DISCARD_ACTIONS_PER_SLOT,
|
PLAY_OR_DISCARD_ACTIONS_PER_SLOT,
|
||||||
DerivedHeuristicConfig,
|
DerivedHeuristicConfig,
|
||||||
SafeHeuristicBot,
|
HeuristicBot,
|
||||||
SafeHeuristicParams,
|
HeuristicParams,
|
||||||
derive_heuristic_config,
|
derive_heuristic_config,
|
||||||
discard_action,
|
discard_action,
|
||||||
draw_from_discard_action,
|
draw_from_discard_action,
|
||||||
@@ -16,8 +16,8 @@ __all__ = [
|
|||||||
"DRAW_FROM_DECK_ACTION",
|
"DRAW_FROM_DECK_ACTION",
|
||||||
"PLAY_OR_DISCARD_ACTIONS_PER_SLOT",
|
"PLAY_OR_DISCARD_ACTIONS_PER_SLOT",
|
||||||
"DerivedHeuristicConfig",
|
"DerivedHeuristicConfig",
|
||||||
"SafeHeuristicBot",
|
"HeuristicBot",
|
||||||
"SafeHeuristicParams",
|
"HeuristicParams",
|
||||||
"derive_heuristic_config",
|
"derive_heuristic_config",
|
||||||
"discard_action",
|
"discard_action",
|
||||||
"draw_from_discard_action",
|
"draw_from_discard_action",
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ def draw_from_discard_action(color: int) -> int:
|
|||||||
return 1 + color
|
return 1 + color
|
||||||
|
|
||||||
|
|
||||||
LOGGER = logging.getLogger("coolrl_lost_cities.games.classic.bots.safe_heuristic")
|
LOGGER = logging.getLogger("coolrl_lost_cities.games.classic.bots.heuristic")
|
||||||
|
|
||||||
|
|
||||||
class _CachedState:
|
class _CachedState:
|
||||||
@@ -66,7 +66,7 @@ class _CachedState:
|
|||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class SafeHeuristicParams:
|
class HeuristicParams:
|
||||||
# Expedition opening.
|
# Expedition opening.
|
||||||
open_target_ratio: float = 0.50
|
open_target_ratio: float = 0.50
|
||||||
open_min_card_ratio: float = 0.40
|
open_min_card_ratio: float = 0.40
|
||||||
@@ -124,7 +124,7 @@ class DerivedHeuristicConfig:
|
|||||||
@lru_cache(maxsize=64)
|
@lru_cache(maxsize=64)
|
||||||
def derive_heuristic_config(
|
def derive_heuristic_config(
|
||||||
config: LostCitiesConfig,
|
config: LostCitiesConfig,
|
||||||
params: SafeHeuristicParams,
|
params: HeuristicParams,
|
||||||
) -> DerivedHeuristicConfig:
|
) -> DerivedHeuristicConfig:
|
||||||
max_color_sum = sum(config.min_rank + rank - 1 for rank in range(1, config.n_ranks + 1))
|
max_color_sum = sum(config.min_rank + rank - 1 for rank in range(1, config.n_ranks + 1))
|
||||||
break_even_sum = -config.expedition_penalty
|
break_even_sum = -config.expedition_penalty
|
||||||
@@ -178,20 +178,20 @@ def derive_heuristic_config(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class SafeHeuristicBot(LostCitiesPolicy):
|
class HeuristicBot(LostCitiesPolicy):
|
||||||
def __init__(self, params: SafeHeuristicParams | None = None):
|
def __init__(self, params: HeuristicParams | None = None):
|
||||||
self.params = params or SafeHeuristicParams()
|
self.params = params or HeuristicParams()
|
||||||
|
|
||||||
def act(self, obs_or_state: PolicyInput) -> int:
|
def act(self, obs_or_state: PolicyInput) -> int:
|
||||||
if not isinstance(obs_or_state, GameState) and not hasattr(obs_or_state, "legal_mask"):
|
if not isinstance(obs_or_state, GameState) and not hasattr(obs_or_state, "legal_mask"):
|
||||||
LOGGER.debug(
|
LOGGER.debug(
|
||||||
"SafeHeuristicBot fallback to first legal: input_type=%s",
|
"HeuristicBot fallback to first legal: input_type=%s",
|
||||||
type(obs_or_state).__name__,
|
type(obs_or_state).__name__,
|
||||||
)
|
)
|
||||||
return first_legal(legal_from_obs(obs_or_state))
|
return first_legal(legal_from_obs(obs_or_state))
|
||||||
|
|
||||||
LOGGER.debug(
|
LOGGER.debug(
|
||||||
"SafeHeuristicBot heuristic path: player=%s phase=%s turn=%s",
|
"HeuristicBot heuristic path: player=%s phase=%s turn=%s",
|
||||||
obs_or_state.current_player,
|
obs_or_state.current_player,
|
||||||
obs_or_state.phase,
|
obs_or_state.phase,
|
||||||
obs_or_state.turn_count,
|
obs_or_state.turn_count,
|
||||||
|
|||||||
@@ -30,11 +30,11 @@ def draw_from_discard_action(color: int) -> int:
|
|||||||
return 1 + color
|
return 1 + color
|
||||||
|
|
||||||
|
|
||||||
LOGGER = logging.getLogger("coolrl_lost_cities.games.classic.bots.safe_heuristic")
|
LOGGER = logging.getLogger("coolrl_lost_cities.games.classic.bots.heuristic")
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class SafeHeuristicParams:
|
class HeuristicParams:
|
||||||
# Expedition opening.
|
# Expedition opening.
|
||||||
open_target_ratio: float = 0.50
|
open_target_ratio: float = 0.50
|
||||||
open_min_card_ratio: float = 0.40
|
open_min_card_ratio: float = 0.40
|
||||||
@@ -92,7 +92,7 @@ class DerivedHeuristicConfig:
|
|||||||
@lru_cache(maxsize=64)
|
@lru_cache(maxsize=64)
|
||||||
def derive_heuristic_config(
|
def derive_heuristic_config(
|
||||||
config: LostCitiesConfig,
|
config: LostCitiesConfig,
|
||||||
params: SafeHeuristicParams,
|
params: HeuristicParams,
|
||||||
) -> DerivedHeuristicConfig:
|
) -> DerivedHeuristicConfig:
|
||||||
max_color_sum = sum(config.min_rank + rank - 1 for rank in range(1, config.n_ranks + 1))
|
max_color_sum = sum(config.min_rank + rank - 1 for rank in range(1, config.n_ranks + 1))
|
||||||
break_even_sum = -config.expedition_penalty
|
break_even_sum = -config.expedition_penalty
|
||||||
@@ -146,20 +146,20 @@ def derive_heuristic_config(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class SafeHeuristicBot(LostCitiesPolicy):
|
class HeuristicBot(LostCitiesPolicy):
|
||||||
def __init__(self, params: SafeHeuristicParams | None = None):
|
def __init__(self, params: HeuristicParams | None = None):
|
||||||
self.params = params or SafeHeuristicParams()
|
self.params = params or HeuristicParams()
|
||||||
|
|
||||||
def act(self, obs_or_state: PolicyInput) -> int:
|
def act(self, obs_or_state: PolicyInput) -> int:
|
||||||
if not isinstance(obs_or_state, GameState) and not hasattr(obs_or_state, "legal_mask"):
|
if not isinstance(obs_or_state, GameState) and not hasattr(obs_or_state, "legal_mask"):
|
||||||
LOGGER.debug(
|
LOGGER.debug(
|
||||||
"SafeHeuristicBot fallback to first legal: input_type=%s",
|
"HeuristicBot fallback to first legal: input_type=%s",
|
||||||
type(obs_or_state).__name__,
|
type(obs_or_state).__name__,
|
||||||
)
|
)
|
||||||
return first_legal(legal_from_obs(obs_or_state))
|
return first_legal(legal_from_obs(obs_or_state))
|
||||||
|
|
||||||
LOGGER.debug(
|
LOGGER.debug(
|
||||||
"SafeHeuristicBot heuristic path: player=%s phase=%s turn=%s",
|
"HeuristicBot heuristic path: player=%s phase=%s turn=%s",
|
||||||
obs_or_state.current_player,
|
obs_or_state.current_player,
|
||||||
obs_or_state.phase,
|
obs_or_state.phase,
|
||||||
obs_or_state.turn_count,
|
obs_or_state.turn_count,
|
||||||
|
|||||||
@@ -3,8 +3,8 @@ from __future__ import annotations
|
|||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
|
|
||||||
from ..policy import LostCitiesPolicy, PolicyInput
|
from ..policy import LostCitiesPolicy, PolicyInput
|
||||||
from .heuristic import SafeHeuristicBot, SafeHeuristicParams
|
from .discard_only import DiscardOnlyBot
|
||||||
from .passive import PassiveDiscardBot
|
from .heuristic import HeuristicBot, HeuristicParams
|
||||||
from .random import RandomBot
|
from .random import RandomBot
|
||||||
|
|
||||||
BotName = str
|
BotName = str
|
||||||
@@ -30,7 +30,7 @@ class NoisyPolicy(LostCitiesPolicy):
|
|||||||
return self.base.act(obs_or_state)
|
return self.base.act(obs_or_state)
|
||||||
|
|
||||||
|
|
||||||
LOOSE_SAFE_HEURISTIC_PARAMS = SafeHeuristicParams(
|
AGGRESSIVE_HEURISTIC_PARAMS = HeuristicParams(
|
||||||
open_target_ratio=0.42,
|
open_target_ratio=0.42,
|
||||||
open_min_card_ratio=0.30,
|
open_min_card_ratio=0.30,
|
||||||
handshake_target_multiplier=1.00,
|
handshake_target_multiplier=1.00,
|
||||||
@@ -38,7 +38,7 @@ LOOSE_SAFE_HEURISTIC_PARAMS = SafeHeuristicParams(
|
|||||||
late_open_block_ratio=0.12,
|
late_open_block_ratio=0.12,
|
||||||
)
|
)
|
||||||
|
|
||||||
STRICT_SAFE_HEURISTIC_PARAMS = SafeHeuristicParams(
|
CAUTIOUS_HEURISTIC_PARAMS = HeuristicParams(
|
||||||
open_target_ratio=0.62,
|
open_target_ratio=0.62,
|
||||||
open_min_card_ratio=0.50,
|
open_min_card_ratio=0.50,
|
||||||
handshake_target_multiplier=1.35,
|
handshake_target_multiplier=1.35,
|
||||||
@@ -49,12 +49,12 @@ STRICT_SAFE_HEURISTIC_PARAMS = SafeHeuristicParams(
|
|||||||
|
|
||||||
BOT_REGISTRY: dict[BotName, PolicyFactory] = {
|
BOT_REGISTRY: dict[BotName, PolicyFactory] = {
|
||||||
DEFAULT_BOT: RandomBot,
|
DEFAULT_BOT: RandomBot,
|
||||||
"passive-discard": lambda seed: PassiveDiscardBot(),
|
"discard-only": lambda seed: DiscardOnlyBot(),
|
||||||
"safe-heuristic": lambda seed: SafeHeuristicBot(),
|
"heuristic-balanced": lambda seed: HeuristicBot(),
|
||||||
"safe-heuristic-loose": lambda seed: SafeHeuristicBot(LOOSE_SAFE_HEURISTIC_PARAMS),
|
"heuristic-aggressive": lambda seed: HeuristicBot(AGGRESSIVE_HEURISTIC_PARAMS),
|
||||||
"safe-heuristic-strict": lambda seed: SafeHeuristicBot(STRICT_SAFE_HEURISTIC_PARAMS),
|
"heuristic-cautious": lambda seed: HeuristicBot(CAUTIOUS_HEURISTIC_PARAMS),
|
||||||
"noisy-safe": lambda seed: NoisyPolicy(
|
"heuristic-noisy": lambda seed: NoisyPolicy(
|
||||||
SafeHeuristicBot(),
|
HeuristicBot(),
|
||||||
RandomBot(seed),
|
RandomBot(seed),
|
||||||
),
|
),
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ class PlotSpec:
|
|||||||
scale: float = 1.0
|
scale: float = 1.0
|
||||||
kind: str = "eval"
|
kind: str = "eval"
|
||||||
fixed_ylim: tuple[float, float] | None = None
|
fixed_ylim: tuple[float, float] | None = None
|
||||||
|
opponents: tuple[str, ...] | None = None
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -28,6 +29,71 @@ class SectionSpec:
|
|||||||
|
|
||||||
|
|
||||||
SECTIONS: tuple[SectionSpec, ...] = (
|
SECTIONS: tuple[SectionSpec, ...] = (
|
||||||
|
SectionSpec(
|
||||||
|
"Core",
|
||||||
|
"analysis_00_core.png",
|
||||||
|
(
|
||||||
|
PlotSpec("Advantage Loss", ("loss/advantage",), "loss", kind="train"),
|
||||||
|
PlotSpec("Strategy Loss", ("loss/strategy",), "loss", kind="train"),
|
||||||
|
PlotSpec(
|
||||||
|
"Avg Score Diff (heuristic_cautious)",
|
||||||
|
("avg_score_diff0",),
|
||||||
|
"score diff",
|
||||||
|
opponents=("heuristic_cautious",),
|
||||||
|
),
|
||||||
|
PlotSpec(
|
||||||
|
"Win Rate (heuristic_cautious)",
|
||||||
|
("win_rate0",),
|
||||||
|
"rate (%)",
|
||||||
|
scale=100.0,
|
||||||
|
fixed_ylim=(0, 100),
|
||||||
|
opponents=("heuristic_cautious",),
|
||||||
|
),
|
||||||
|
PlotSpec(
|
||||||
|
"Win Rate (random)",
|
||||||
|
("win_rate0",),
|
||||||
|
"rate (%)",
|
||||||
|
scale=100.0,
|
||||||
|
fixed_ylim=(0, 100),
|
||||||
|
opponents=("random",),
|
||||||
|
),
|
||||||
|
PlotSpec(
|
||||||
|
"Avg Opened Colors (heuristic_cautious)",
|
||||||
|
("avg_opened_colors",),
|
||||||
|
"colors",
|
||||||
|
fixed_ylim=(0, 5),
|
||||||
|
opponents=("heuristic_cautious",),
|
||||||
|
),
|
||||||
|
PlotSpec(
|
||||||
|
"Positive Expedition Rate (heuristic_cautious)",
|
||||||
|
("positive_expedition_rate",),
|
||||||
|
"rate (%)",
|
||||||
|
scale=100.0,
|
||||||
|
fixed_ylim=(0, 100),
|
||||||
|
opponents=("heuristic_cautious",),
|
||||||
|
),
|
||||||
|
PlotSpec(
|
||||||
|
"Bonus Expedition Rate (heuristic_cautious)",
|
||||||
|
("bonus_expedition_rate",),
|
||||||
|
"rate (%)",
|
||||||
|
scale=100.0,
|
||||||
|
fixed_ylim=(0, 100),
|
||||||
|
opponents=("heuristic_cautious",),
|
||||||
|
),
|
||||||
|
PlotSpec(
|
||||||
|
"Score per Opened Color (heuristic_cautious)",
|
||||||
|
("score_per_opened_color",),
|
||||||
|
"score / color",
|
||||||
|
opponents=("heuristic_cautious",),
|
||||||
|
),
|
||||||
|
PlotSpec(
|
||||||
|
"Policy Entropy (heuristic_cautious)",
|
||||||
|
("policy_entropy",),
|
||||||
|
"entropy",
|
||||||
|
opponents=("heuristic_cautious",),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
),
|
||||||
SectionSpec(
|
SectionSpec(
|
||||||
"Loss",
|
"Loss",
|
||||||
"analysis_01_loss.png",
|
"analysis_01_loss.png",
|
||||||
@@ -107,48 +173,6 @@ SECTIONS: tuple[SectionSpec, ...] = (
|
|||||||
PlotSpec("Expedition Cards", ("avg_expedition_cards",), "cards"),
|
PlotSpec("Expedition Cards", ("avg_expedition_cards",), "cards"),
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
SectionSpec(
|
|
||||||
"OpenQuality",
|
|
||||||
"analysis_05_open_quality.png",
|
|
||||||
(
|
|
||||||
PlotSpec(
|
|
||||||
"Bad Open Rate",
|
|
||||||
("bad_open_rate",),
|
|
||||||
"rate (%)",
|
|
||||||
scale=100.0,
|
|
||||||
fixed_ylim=(0, 100),
|
|
||||||
),
|
|
||||||
PlotSpec(
|
|
||||||
"Weak Open Rate",
|
|
||||||
("weak_open_rate",),
|
|
||||||
"rate (%)",
|
|
||||||
scale=100.0,
|
|
||||||
fixed_ylim=(0, 100),
|
|
||||||
),
|
|
||||||
PlotSpec(
|
|
||||||
"Bad or Weak Open Rate",
|
|
||||||
("bad_or_weak_open_rate",),
|
|
||||||
"rate (%)",
|
|
||||||
scale=100.0,
|
|
||||||
fixed_ylim=(0, 100),
|
|
||||||
),
|
|
||||||
PlotSpec(
|
|
||||||
"Good Open Rate",
|
|
||||||
("good_open_rate",),
|
|
||||||
"rate (%)",
|
|
||||||
scale=100.0,
|
|
||||||
fixed_ylim=(0, 100),
|
|
||||||
),
|
|
||||||
PlotSpec("Bad Open per Game", ("bad_open_per_game",), "opens / game"),
|
|
||||||
PlotSpec(
|
|
||||||
"Bad or Weak Open per Game",
|
|
||||||
("bad_or_weak_open_per_game",),
|
|
||||||
"opens / game",
|
|
||||||
),
|
|
||||||
PlotSpec("Opening Play Actions", ("opening_play_actions",), "actions / game"),
|
|
||||||
PlotSpec("Opening Recoverable p25", ("opening_recoverable_score_p25",), "score"),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
SectionSpec(
|
SectionSpec(
|
||||||
"ExpeditionOutcomes",
|
"ExpeditionOutcomes",
|
||||||
"analysis_06_expedition_outcomes.png",
|
"analysis_06_expedition_outcomes.png",
|
||||||
@@ -188,23 +212,6 @@ SECTIONS: tuple[SectionSpec, ...] = (
|
|||||||
PlotSpec("Score per Opened Color", ("score_per_opened_color",), "score / color"),
|
PlotSpec("Score per Opened Color", ("score_per_opened_color",), "score / color"),
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
SectionSpec(
|
|
||||||
"Calibration",
|
|
||||||
"analysis_07_calibration.png",
|
|
||||||
(
|
|
||||||
PlotSpec(
|
|
||||||
"First Open Recoverable Score",
|
|
||||||
(
|
|
||||||
"first_open_recoverable_score_mean_for_positive_final",
|
|
||||||
"first_open_recoverable_score_mean_for_negative_final",
|
|
||||||
),
|
|
||||||
"score",
|
|
||||||
),
|
|
||||||
PlotSpec("Opening Recoverable Mean", ("opening_recoverable_score_mean",), "score"),
|
|
||||||
PlotSpec("Opening Recoverable p25", ("opening_recoverable_score_p25",), "score"),
|
|
||||||
PlotSpec("Calibration Gap", ("calibration_gap",), "score"),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
SectionSpec(
|
SectionSpec(
|
||||||
"Traversal",
|
"Traversal",
|
||||||
"analysis_08_traversal.png",
|
"analysis_08_traversal.png",
|
||||||
@@ -311,41 +318,23 @@ SECTIONS: tuple[SectionSpec, ...] = (
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
SELECTIVITY_PLOTS: tuple[PlotSpec, ...] = (
|
|
||||||
PlotSpec("Opened Colors", ("avg_opened_colors",), "colors"),
|
|
||||||
PlotSpec("5-Color Open Count", ("5_color_open_count",), "games / eval"),
|
|
||||||
PlotSpec("Opening Recoverable Mean", ("opening_recoverable_score_mean",), "score"),
|
|
||||||
PlotSpec("Calibration Gap", ("calibration_gap",), "score"),
|
|
||||||
PlotSpec(
|
|
||||||
"First Open Recoverable Score",
|
|
||||||
(
|
|
||||||
"first_open_recoverable_score_mean_for_positive_final",
|
|
||||||
"first_open_recoverable_score_mean_for_negative_final",
|
|
||||||
),
|
|
||||||
"score",
|
|
||||||
),
|
|
||||||
PlotSpec("Opening Play Actions", ("opening_play_actions",), "actions / game"),
|
|
||||||
)
|
|
||||||
|
|
||||||
SUMMARY_EVAL_METRICS: tuple[tuple[str, str, float], ...] = (
|
SUMMARY_EVAL_METRICS: tuple[tuple[str, str, float], ...] = (
|
||||||
("win_rate0", "win rate (%)", 100.0),
|
("win_rate0", "win rate (%)", 100.0),
|
||||||
("avg_score_diff0", "avg score diff", 1.0),
|
("avg_score_diff0", "avg score diff", 1.0),
|
||||||
("avg_score0", "avg score", 1.0),
|
("avg_score0", "avg score", 1.0),
|
||||||
("play_action_rate", "play rate (%)", 100.0),
|
("play_action_rate", "play rate (%)", 100.0),
|
||||||
("avg_opened_colors", "opened colors", 1.0),
|
("avg_opened_colors", "opened colors", 1.0),
|
||||||
("bad_open_rate", "bad open (%)", 100.0),
|
|
||||||
("score_per_opened_color", "score / opened color", 1.0),
|
("score_per_opened_color", "score / opened color", 1.0),
|
||||||
("calibration_gap", "calibration gap", 1.0),
|
|
||||||
("bonus_contribution_per_game", "bonus / game", 1.0),
|
("bonus_contribution_per_game", "bonus / game", 1.0),
|
||||||
)
|
)
|
||||||
|
|
||||||
OPPONENT_COLORS: dict[str, str] = {
|
OPPONENT_COLORS: dict[str, str] = {
|
||||||
"noisy_safe": "tab:blue",
|
"heuristic_noisy": "tab:blue",
|
||||||
"passive_discard": "tab:orange",
|
"discard_only": "tab:orange",
|
||||||
"random": "tab:green",
|
"random": "tab:green",
|
||||||
"safe_heuristic": "tab:red",
|
"heuristic_balanced": "tab:red",
|
||||||
"safe_heuristic_loose": "tab:purple",
|
"heuristic_aggressive": "tab:purple",
|
||||||
"safe_heuristic_strict": "tab:brown",
|
"heuristic_cautious": "tab:brown",
|
||||||
}
|
}
|
||||||
|
|
||||||
TRAVERSAL_COLORS: dict[str, str] = {
|
TRAVERSAL_COLORS: dict[str, str] = {
|
||||||
@@ -428,7 +417,14 @@ def plot_section(
|
|||||||
if spec.kind == "train":
|
if spec.kind == "train":
|
||||||
plotted = _plot_train_spec(ax, rows, spec, smoothing_window=smoothing_window)
|
plotted = _plot_train_spec(ax, rows, spec, smoothing_window=smoothing_window)
|
||||||
else:
|
else:
|
||||||
plotted = _plot_eval_spec(ax, rows, opponents, spec, smoothing_window=smoothing_window)
|
filtered_opponents = (
|
||||||
|
[o for o in opponents if o in spec.opponents]
|
||||||
|
if spec.opponents is not None
|
||||||
|
else opponents
|
||||||
|
)
|
||||||
|
plotted = _plot_eval_spec(
|
||||||
|
ax, rows, filtered_opponents, spec, smoothing_window=smoothing_window
|
||||||
|
)
|
||||||
_finish_axis(
|
_finish_axis(
|
||||||
ax, spec.title, ylabel=spec.ylabel, plotted=plotted, fixed_ylim=spec.fixed_ylim
|
ax, spec.title, ylabel=spec.ylabel, plotted=plotted, fixed_ylim=spec.fixed_ylim
|
||||||
)
|
)
|
||||||
@@ -539,12 +535,6 @@ def analyze_run(
|
|||||||
if plot_section(rows, section, path, smoothing_window=smoothing_window):
|
if plot_section(rows, section, path, smoothing_window=smoothing_window):
|
||||||
written.append(path)
|
written.append(path)
|
||||||
|
|
||||||
selectivity_path = output_dir / _with_filename_suffix(
|
|
||||||
"analysis_09_selectivity.png", filename_suffix
|
|
||||||
)
|
|
||||||
if plot_selectivity(rows, selectivity_path, smoothing_window=smoothing_window):
|
|
||||||
written.append(selectivity_path)
|
|
||||||
|
|
||||||
final_eval_path = output_dir / _with_filename_suffix(
|
final_eval_path = output_dir / _with_filename_suffix(
|
||||||
"analysis_final_eval_summary.png", filename_suffix
|
"analysis_final_eval_summary.png", filename_suffix
|
||||||
)
|
)
|
||||||
@@ -566,45 +556,6 @@ def _with_filename_suffix(filename: str, suffix: str) -> str:
|
|||||||
return f"{path.stem}{suffix}{path.suffix}"
|
return f"{path.stem}{suffix}{path.suffix}"
|
||||||
|
|
||||||
|
|
||||||
def plot_selectivity(
|
|
||||||
rows: list[dict[str, Any]],
|
|
||||||
output: Path,
|
|
||||||
*,
|
|
||||||
smoothing_window: int,
|
|
||||||
) -> bool:
|
|
||||||
import matplotlib.pyplot as plt
|
|
||||||
|
|
||||||
opponents = opponent_names(rows)
|
|
||||||
if not opponents:
|
|
||||||
return False
|
|
||||||
|
|
||||||
fig, axes = plt.subplots(3, 2, figsize=(16, 12), squeeze=False)
|
|
||||||
axes_flat = list(axes.flat)
|
|
||||||
plotted_any = False
|
|
||||||
|
|
||||||
for ax, spec in zip(axes_flat, SELECTIVITY_PLOTS, strict=True):
|
|
||||||
plotted = _plot_eval_spec(ax, rows, opponents, spec, smoothing_window=smoothing_window)
|
|
||||||
_finish_axis(ax, spec.title, ylabel=spec.ylabel, plotted=plotted)
|
|
||||||
plotted_any = plotted_any or plotted
|
|
||||||
|
|
||||||
handles, labels = _legend_items(axes_flat)
|
|
||||||
if handles:
|
|
||||||
fig.legend(handles, labels, loc="upper center", ncols=min(len(labels), 6), fontsize="small")
|
|
||||||
suffix = f" ({smoothing_window}-iter moving average)" if smoothing_window > 1 else ""
|
|
||||||
fig.suptitle(
|
|
||||||
f"Lost Cities Deep CFR selectivity metrics{suffix}",
|
|
||||||
fontsize=14,
|
|
||||||
fontweight="bold",
|
|
||||||
)
|
|
||||||
fig.tight_layout(rect=(0, 0, 1, 0.95))
|
|
||||||
if not plotted_any:
|
|
||||||
plt.close(fig)
|
|
||||||
return False
|
|
||||||
fig.savefig(output, dpi=150)
|
|
||||||
plt.close(fig)
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def _all_eval_metrics() -> set[str]:
|
def _all_eval_metrics() -> set[str]:
|
||||||
metrics: set[str] = set()
|
metrics: set[str] = set()
|
||||||
for section in SECTIONS:
|
for section in SECTIONS:
|
||||||
|
|||||||
@@ -143,16 +143,16 @@ class TraversalConfig(StrictModel):
|
|||||||
@field_validator("cutoff_rollout_policy")
|
@field_validator("cutoff_rollout_policy")
|
||||||
@classmethod
|
@classmethod
|
||||||
def _validate_cutoff_rollout_policy(cls, value: str) -> str:
|
def _validate_cutoff_rollout_policy(cls, value: str) -> str:
|
||||||
if value not in {"random", "safe_heuristic"}:
|
if value not in {"random", "heuristic_balanced"}:
|
||||||
raise ValueError("must be 'random' or 'safe_heuristic'")
|
raise ValueError("must be 'random' or 'heuristic_balanced'")
|
||||||
return value
|
return value
|
||||||
|
|
||||||
@field_validator("opponent_policy")
|
@field_validator("opponent_policy")
|
||||||
@classmethod
|
@classmethod
|
||||||
def _validate_opponent_policy(cls, value: str) -> str:
|
def _validate_opponent_policy(cls, value: str) -> str:
|
||||||
if value not in {"network", "safe_heuristic", "self_play_league", "average_strategy"}:
|
if value not in {"network", "heuristic_balanced", "self_play_league", "average_strategy"}:
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
"must be 'network', 'safe_heuristic', 'self_play_league', or 'average_strategy'"
|
"must be 'network', 'heuristic_balanced', 'self_play_league', or 'average_strategy'"
|
||||||
)
|
)
|
||||||
return value
|
return value
|
||||||
|
|
||||||
@@ -294,13 +294,35 @@ class CheckpointConfig(StrictModel):
|
|||||||
class EvaluationConfig(StrictModel):
|
class EvaluationConfig(StrictModel):
|
||||||
eval_every: int = 50
|
eval_every: int = 50
|
||||||
games: int = 10
|
games: int = 10
|
||||||
opponents: tuple[str, ...] = ("random", "safe_heuristic")
|
opponents: tuple[str, ...] = ("random", "heuristic_balanced")
|
||||||
|
extended_eval_every: int = 0
|
||||||
|
extended_opponents: tuple[str, ...] = ()
|
||||||
max_steps: int = 10_000
|
max_steps: int = 10_000
|
||||||
on_max_steps: str = "score_diff"
|
on_max_steps: str = "score_diff"
|
||||||
batch_size: int = 64
|
batch_size: int = 64
|
||||||
device: str = "trainer"
|
device: str = "trainer"
|
||||||
num_workers: int = 4
|
num_workers: int = 4
|
||||||
|
|
||||||
|
def opponents_for_iteration(self, iteration: int) -> tuple[str, ...]:
|
||||||
|
"""Return the opponent set to evaluate against at ``iteration``.
|
||||||
|
|
||||||
|
- Always returns ``opponents`` when ``iteration % eval_every == 0``.
|
||||||
|
- When ``extended_eval_every > 0`` and ``iteration`` is also a multiple
|
||||||
|
of ``extended_eval_every``, appends ``extended_opponents`` (de-duplicated)
|
||||||
|
to the core list.
|
||||||
|
- Returns ``()`` when no eval is scheduled this iteration.
|
||||||
|
"""
|
||||||
|
if self.eval_every <= 0 or iteration <= 0:
|
||||||
|
return ()
|
||||||
|
if iteration % self.eval_every != 0:
|
||||||
|
return ()
|
||||||
|
result = list(self.opponents)
|
||||||
|
if self.extended_eval_every > 0 and iteration % self.extended_eval_every == 0:
|
||||||
|
for opponent in self.extended_opponents:
|
||||||
|
if opponent not in result:
|
||||||
|
result.append(opponent)
|
||||||
|
return tuple(result)
|
||||||
|
|
||||||
@field_validator("on_max_steps")
|
@field_validator("on_max_steps")
|
||||||
@classmethod
|
@classmethod
|
||||||
def _validate_on_max_steps(cls, value: str) -> str:
|
def _validate_on_max_steps(cls, value: str) -> str:
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ import numpy as np
|
|||||||
import torch
|
import torch
|
||||||
from torch import nn
|
from torch import nn
|
||||||
|
|
||||||
from coolrl_lost_cities.games.classic.bots import SafeHeuristicBot
|
from coolrl_lost_cities.games.classic.bots import HeuristicBot
|
||||||
from coolrl_lost_cities.games.classic.deep_cfr.encoding import encode_info_state, input_dim
|
from coolrl_lost_cities.games.classic.deep_cfr.encoding import encode_info_state, input_dim
|
||||||
from coolrl_lost_cities.games.classic.deep_cfr.networks import DeepCFRMLP
|
from coolrl_lost_cities.games.classic.deep_cfr.networks import DeepCFRMLP
|
||||||
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig, classic_config
|
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig, classic_config
|
||||||
@@ -18,7 +18,7 @@ class ImitationMetrics:
|
|||||||
loss: float
|
loss: float
|
||||||
|
|
||||||
|
|
||||||
def collect_safe_heuristic_samples(
|
def collect_heuristic_samples(
|
||||||
config: LostCitiesConfig | None = None,
|
config: LostCitiesConfig | None = None,
|
||||||
*,
|
*,
|
||||||
games: int = 4,
|
games: int = 4,
|
||||||
@@ -28,7 +28,7 @@ def collect_safe_heuristic_samples(
|
|||||||
game_config = config or classic_config(seed=seed)
|
game_config = config or classic_config(seed=seed)
|
||||||
probe = GameState.new_game(game_config, seed=seed)
|
probe = GameState.new_game(game_config, seed=seed)
|
||||||
action_size = 2 * probe.config.hand_size + 1 + probe.config.n_colors
|
action_size = 2 * probe.config.hand_size + 1 + probe.config.n_colors
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
infos: list[np.ndarray] = []
|
infos: list[np.ndarray] = []
|
||||||
targets: list[np.ndarray] = []
|
targets: list[np.ndarray] = []
|
||||||
masks: list[np.ndarray] = []
|
masks: list[np.ndarray] = []
|
||||||
@@ -67,7 +67,7 @@ def pretrain_strategy_network(
|
|||||||
learning_rate: float = 1.0e-3,
|
learning_rate: float = 1.0e-3,
|
||||||
device: torch.device | str = "cpu",
|
device: torch.device | str = "cpu",
|
||||||
) -> ImitationMetrics:
|
) -> ImitationMetrics:
|
||||||
x_np, y_np, legal_np = collect_safe_heuristic_samples(config, games=games, seed=seed)
|
x_np, y_np, legal_np = collect_heuristic_samples(config, games=games, seed=seed)
|
||||||
device = torch.device(device)
|
device = torch.device(device)
|
||||||
strategy_network.to(device)
|
strategy_network.to(device)
|
||||||
strategy_network.train()
|
strategy_network.train()
|
||||||
|
|||||||
@@ -894,17 +894,15 @@ class DeepCFRTrainer:
|
|||||||
self.tracker.log_event(_format_iteration_summary(metrics, data))
|
self.tracker.log_event(_format_iteration_summary(metrics, data))
|
||||||
|
|
||||||
def _evaluate(self, iteration: int) -> dict[str, float | int]:
|
def _evaluate(self, iteration: int) -> dict[str, float | int]:
|
||||||
if (
|
opponents = self.config.evaluation.opponents_for_iteration(iteration)
|
||||||
self.config.evaluation.eval_every <= 0
|
if not opponents:
|
||||||
or iteration % self.config.evaluation.eval_every != 0
|
|
||||||
):
|
|
||||||
return {}
|
return {}
|
||||||
results: dict[str, float | int] = {}
|
results: dict[str, float | int] = {}
|
||||||
eval_device = self._evaluation_device()
|
eval_device = self._evaluation_device()
|
||||||
if self.config.evaluation.resolved_num_workers(len(self.config.evaluation.opponents)) > 1:
|
if self.config.evaluation.resolved_num_workers(len(opponents)) > 1:
|
||||||
return self._evaluate_parallel(iteration, eval_device)
|
return self._evaluate_parallel(iteration, eval_device, opponents)
|
||||||
eval_network = self._evaluation_network(eval_device)
|
eval_network = self._evaluation_network(eval_device)
|
||||||
for opponent in self.config.evaluation.opponents:
|
for opponent in opponents:
|
||||||
result = evaluate_strategy_network(
|
result = evaluate_strategy_network(
|
||||||
eval_network,
|
eval_network,
|
||||||
self.game_config,
|
self.game_config,
|
||||||
@@ -925,8 +923,8 @@ class DeepCFRTrainer:
|
|||||||
self,
|
self,
|
||||||
iteration: int,
|
iteration: int,
|
||||||
eval_device: torch.device,
|
eval_device: torch.device,
|
||||||
|
opponents: tuple[str, ...],
|
||||||
) -> dict[str, float | int]:
|
) -> dict[str, float | int]:
|
||||||
opponents = self.config.evaluation.opponents
|
|
||||||
max_workers = self.config.evaluation.resolved_num_workers(len(opponents))
|
max_workers = self.config.evaluation.resolved_num_workers(len(opponents))
|
||||||
self.tracker.log_event(
|
self.tracker.log_event(
|
||||||
f"Evaluation multiprocessing enabled iteration={iteration} "
|
f"Evaluation multiprocessing enabled iteration={iteration} "
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ from libc.stdlib cimport free, malloc
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
import torch
|
import torch
|
||||||
|
|
||||||
from coolrl_lost_cities.games.classic.bots import SafeHeuristicBot
|
from coolrl_lost_cities.games.classic.bots import HeuristicBot
|
||||||
from coolrl_lost_cities.games.classic.deep_cfr.cfr_math cimport regret_matching_c
|
from coolrl_lost_cities.games.classic.deep_cfr.cfr_math cimport regret_matching_c
|
||||||
from coolrl_lost_cities.games.classic.deep_cfr.encoding cimport (
|
from coolrl_lost_cities.games.classic.deep_cfr.encoding cimport (
|
||||||
_encode_info_state_with_flags_c,
|
_encode_info_state_with_flags_c,
|
||||||
@@ -70,8 +70,8 @@ cdef class CythonDeepCFRTraverser:
|
|||||||
cdef object device
|
cdef object device
|
||||||
cdef object encoding
|
cdef object encoding
|
||||||
cdef object league_advantage_networks
|
cdef object league_advantage_networks
|
||||||
cdef object safe_heuristic_rollout_bot
|
cdef object heuristic_rollout_bot
|
||||||
cdef object safe_heuristic_opponent_bot
|
cdef object heuristic_opponent_bot
|
||||||
cdef int action_size
|
cdef int action_size
|
||||||
cdef int input_dim
|
cdef int input_dim
|
||||||
cdef float epsilon
|
cdef float epsilon
|
||||||
@@ -185,15 +185,15 @@ cdef class CythonDeepCFRTraverser:
|
|||||||
raise ValueError("cutoff_value_mode must be 'score_diff' or 'random_rollout'")
|
raise ValueError("cutoff_value_mode must be 'score_diff' or 'random_rollout'")
|
||||||
self.cutoff_random_rollout = cutoff_value_mode == "random_rollout"
|
self.cutoff_random_rollout = cutoff_value_mode == "random_rollout"
|
||||||
self.cutoff_rollouts = max(0, cutoff_rollouts)
|
self.cutoff_rollouts = max(0, cutoff_rollouts)
|
||||||
if cutoff_rollout_policy not in {"random", "safe_heuristic"}:
|
if cutoff_rollout_policy not in {"random", "heuristic_balanced"}:
|
||||||
raise ValueError("cutoff_rollout_policy must be 'random' or 'safe_heuristic'")
|
raise ValueError("cutoff_rollout_policy must be 'random' or 'heuristic_balanced'")
|
||||||
self.cutoff_rollout_max_steps = max(1, cutoff_rollout_max_steps)
|
self.cutoff_rollout_max_steps = max(1, cutoff_rollout_max_steps)
|
||||||
self.safe_heuristic_rollout_bot = (
|
self.heuristic_rollout_bot = (
|
||||||
SafeHeuristicBot() if cutoff_rollout_policy == "safe_heuristic" else None
|
HeuristicBot() if cutoff_rollout_policy == "heuristic_balanced" else None
|
||||||
)
|
)
|
||||||
if opponent_policy == "network":
|
if opponent_policy == "network":
|
||||||
self.opponent_policy_id = 0
|
self.opponent_policy_id = 0
|
||||||
elif opponent_policy == "safe_heuristic":
|
elif opponent_policy == "heuristic_balanced":
|
||||||
self.opponent_policy_id = 1
|
self.opponent_policy_id = 1
|
||||||
elif opponent_policy == "self_play_league":
|
elif opponent_policy == "self_play_league":
|
||||||
self.opponent_policy_id = 2
|
self.opponent_policy_id = 2
|
||||||
@@ -205,7 +205,7 @@ cdef class CythonDeepCFRTraverser:
|
|||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
"opponent_policy must be 'network', 'safe_heuristic', 'self_play_league', or 'average_strategy'"
|
"opponent_policy must be 'network', 'heuristic_balanced', 'self_play_league', or 'average_strategy'"
|
||||||
)
|
)
|
||||||
if all_negative_fallback == "uniform":
|
if all_negative_fallback == "uniform":
|
||||||
self.all_negative_fallback_id = 0
|
self.all_negative_fallback_id = 0
|
||||||
@@ -225,8 +225,8 @@ cdef class CythonDeepCFRTraverser:
|
|||||||
self.self_play_recent_window = max(0, self_play_recent_window)
|
self.self_play_recent_window = max(0, self_play_recent_window)
|
||||||
self.active_self_play_bucket = 0
|
self.active_self_play_bucket = 0
|
||||||
self.active_self_play_networks = None
|
self.active_self_play_networks = None
|
||||||
self.safe_heuristic_opponent_bot = (
|
self.heuristic_opponent_bot = (
|
||||||
SafeHeuristicBot()
|
HeuristicBot()
|
||||||
if self.opponent_policy_id == 1 or self.self_play_anchor_probability > 0.0
|
if self.opponent_policy_id == 1 or self.self_play_anchor_probability > 0.0
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
@@ -630,9 +630,9 @@ cdef class CythonDeepCFRTraverser:
|
|||||||
if player == traverser or self.opponent_policy_id == 0:
|
if player == traverser or self.opponent_policy_id == 0:
|
||||||
return -1
|
return -1
|
||||||
if self.opponent_policy_id == 1:
|
if self.opponent_policy_id == 1:
|
||||||
if self.safe_heuristic_opponent_bot is None:
|
if self.heuristic_opponent_bot is None:
|
||||||
self.safe_heuristic_opponent_bot = SafeHeuristicBot()
|
self.heuristic_opponent_bot = HeuristicBot()
|
||||||
return int(self.safe_heuristic_opponent_bot.act(state))
|
return int(self.heuristic_opponent_bot.act(state))
|
||||||
if self.opponent_policy_id == 3:
|
if self.opponent_policy_id == 3:
|
||||||
self._policy_from_strategy_network(state, player, legal, policy)
|
self._policy_from_strategy_network(state, player, legal, policy)
|
||||||
for i in range(self.action_size):
|
for i in range(self.action_size):
|
||||||
@@ -649,9 +649,9 @@ cdef class CythonDeepCFRTraverser:
|
|||||||
if bucket == 0:
|
if bucket == 0:
|
||||||
return -1
|
return -1
|
||||||
if bucket == 3:
|
if bucket == 3:
|
||||||
if self.safe_heuristic_opponent_bot is None:
|
if self.heuristic_opponent_bot is None:
|
||||||
self.safe_heuristic_opponent_bot = SafeHeuristicBot()
|
self.heuristic_opponent_bot = HeuristicBot()
|
||||||
return int(self.safe_heuristic_opponent_bot.act(state))
|
return int(self.heuristic_opponent_bot.act(state))
|
||||||
networks = self.active_self_play_networks
|
networks = self.active_self_play_networks
|
||||||
if networks is None:
|
if networks is None:
|
||||||
return -1
|
return -1
|
||||||
@@ -839,8 +839,8 @@ cdef class CythonDeepCFRTraverser:
|
|||||||
if swapped_indices == NULL:
|
if swapped_indices == NULL:
|
||||||
raise MemoryError()
|
raise MemoryError()
|
||||||
while not state.terminal and steps < self.cutoff_rollout_max_steps:
|
while not state.terminal and steps < self.cutoff_rollout_max_steps:
|
||||||
if self.safe_heuristic_rollout_bot is not None:
|
if self.heuristic_rollout_bot is not None:
|
||||||
local_action = int(self.safe_heuristic_rollout_bot.act(state))
|
local_action = int(self.heuristic_rollout_bot.act(state))
|
||||||
unified_action = self._to_unified_action_c(state, local_action)
|
unified_action = self._to_unified_action_c(state, local_action)
|
||||||
else:
|
else:
|
||||||
count = state._unified_legal_actions_c(actions)
|
count = state._unified_legal_actions_c(actions)
|
||||||
|
|||||||
@@ -277,7 +277,7 @@ def evaluate_policy(
|
|||||||
|
|
||||||
def main(argv: list[str] | None = None) -> None:
|
def main(argv: list[str] | None = None) -> None:
|
||||||
parser = argparse.ArgumentParser(description="Evaluate Lost Cities classic bots.")
|
parser = argparse.ArgumentParser(description="Evaluate Lost Cities classic bots.")
|
||||||
parser.add_argument("--bot0", default="safe-heuristic", choices=available_bot_names())
|
parser.add_argument("--bot0", default="heuristic-balanced", choices=available_bot_names())
|
||||||
parser.add_argument("--bot1", default="random", choices=available_bot_names())
|
parser.add_argument("--bot1", default="random", choices=available_bot_names())
|
||||||
parser.add_argument("--games", type=int, default=100)
|
parser.add_argument("--games", type=int, default=100)
|
||||||
parser.add_argument("--seed", type=int, default=1)
|
parser.add_argument("--seed", type=int, default=1)
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
from coolrl_lost_cities.games.classic.game import Card, GameState, LostCitiesConfig
|
from coolrl_lost_cities.games.classic.game import Card, GameState, LostCitiesConfig
|
||||||
|
|
||||||
from coolrl_lost_cities.games.classic.bots import (
|
from coolrl_lost_cities.games.classic.bots import (
|
||||||
|
HeuristicBot,
|
||||||
LostCitiesPolicy,
|
LostCitiesPolicy,
|
||||||
RandomBot,
|
RandomBot,
|
||||||
SafeHeuristicBot,
|
|
||||||
)
|
)
|
||||||
from coolrl_lost_cities.games.classic.bots.heuristic import draw_from_discard_action
|
from coolrl_lost_cities.games.classic.bots.heuristic import draw_from_discard_action
|
||||||
from coolrl_lost_cities.games.classic.evaluation import play_game_for_evaluation
|
from coolrl_lost_cities.games.classic.evaluation import play_game_for_evaluation
|
||||||
@@ -19,13 +19,13 @@ def _expeditions(config: LostCitiesConfig) -> list[list[list[Card]]]:
|
|||||||
|
|
||||||
def test_builtin_bots_implement_lost_cities_policy() -> None:
|
def test_builtin_bots_implement_lost_cities_policy() -> None:
|
||||||
assert isinstance(RandomBot(1), LostCitiesPolicy)
|
assert isinstance(RandomBot(1), LostCitiesPolicy)
|
||||||
assert isinstance(SafeHeuristicBot(), LostCitiesPolicy)
|
assert isinstance(HeuristicBot(), LostCitiesPolicy)
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_mirror_match_finishes() -> None:
|
def test_heuristic_mirror_match_finishes() -> None:
|
||||||
state, result = play_game_for_evaluation(
|
state, result = play_game_for_evaluation(
|
||||||
SafeHeuristicBot(),
|
HeuristicBot(),
|
||||||
SafeHeuristicBot(),
|
HeuristicBot(),
|
||||||
LostCitiesConfig(n_colors=3, n_ranks=5, n_handshakes=1, hand_size=5),
|
LostCitiesConfig(n_colors=3, n_ranks=5, n_handshakes=1, hand_size=5),
|
||||||
seed=2000,
|
seed=2000,
|
||||||
max_steps=200,
|
max_steps=200,
|
||||||
@@ -34,9 +34,9 @@ def test_safe_heuristic_mirror_match_finishes() -> None:
|
|||||||
assert result.timed_out is False
|
assert result.timed_out is False
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_opponent_value_ignores_hidden_hand() -> None:
|
def test_heuristic_opponent_value_ignores_hidden_hand() -> None:
|
||||||
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
|
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
discard_card = Card(color=0, rank=6)
|
discard_card = Card(color=0, rank=6)
|
||||||
|
|
||||||
expeditions_a = _expeditions(config)
|
expeditions_a = _expeditions(config)
|
||||||
@@ -76,9 +76,9 @@ def test_safe_heuristic_opponent_value_ignores_hidden_hand() -> None:
|
|||||||
assert value_a == value_b
|
assert value_a == value_b
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_started_expedition_value_ignores_invalid_lower_followup() -> None:
|
def test_heuristic_started_expedition_value_ignores_invalid_lower_followup() -> None:
|
||||||
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
|
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
high_card = Card(color=0, rank=8)
|
high_card = Card(color=0, rank=8)
|
||||||
|
|
||||||
base_expeditions = _expeditions(config)
|
base_expeditions = _expeditions(config)
|
||||||
@@ -115,9 +115,9 @@ def test_safe_heuristic_started_expedition_value_ignores_invalid_lower_followup(
|
|||||||
assert lower_followup_value == base_value
|
assert lower_followup_value == base_value
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_draws_playable_discard_instead_of_deck() -> None:
|
def test_heuristic_draws_playable_discard_instead_of_deck() -> None:
|
||||||
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
|
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=3)
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
|
|
||||||
expeditions = _expeditions(config)
|
expeditions = _expeditions(config)
|
||||||
expeditions[0][0] = [Card(color=0, rank=4)]
|
expeditions[0][0] = [Card(color=0, rank=4)]
|
||||||
@@ -132,9 +132,9 @@ def test_safe_heuristic_draws_playable_discard_instead_of_deck() -> None:
|
|||||||
assert bot._act_draw(state) == draw_from_discard_action(0)
|
assert bot._act_draw(state) == draw_from_discard_action(0)
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_can_draw_discard_to_deny_opponent_when_losing() -> None:
|
def test_heuristic_can_draw_discard_to_deny_opponent_when_losing() -> None:
|
||||||
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=4)
|
config = LostCitiesConfig(n_colors=2, n_ranks=8, hand_size=4)
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
|
|
||||||
expeditions = _expeditions(config)
|
expeditions = _expeditions(config)
|
||||||
expeditions[0][1] = [Card(color=1, rank=8)]
|
expeditions[0][1] = [Card(color=1, rank=8)]
|
||||||
@@ -158,9 +158,9 @@ def test_safe_heuristic_can_draw_discard_to_deny_opponent_when_losing() -> None:
|
|||||||
assert bot._act_draw(state) == draw_from_discard_action(0)
|
assert bot._act_draw(state) == draw_from_discard_action(0)
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_classic_self_play_opens_expeditions() -> None:
|
def test_heuristic_classic_self_play_opens_expeditions() -> None:
|
||||||
state = GameState.new_game(LostCitiesConfig(), seed=1)
|
state = GameState.new_game(LostCitiesConfig(), seed=1)
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
player0_actions: list[int] = []
|
player0_actions: list[int] = []
|
||||||
|
|
||||||
for _ in range(60):
|
for _ in range(60):
|
||||||
@@ -182,9 +182,9 @@ def test_safe_heuristic_classic_self_play_opens_expeditions() -> None:
|
|||||||
assert any(state.expeditions[0][color] for color in range(state.config.n_colors))
|
assert any(state.expeditions[0][color] for color in range(state.config.n_colors))
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_avoids_opening_weak_fifth_color() -> None:
|
def test_heuristic_avoids_opening_weak_fifth_color() -> None:
|
||||||
config = LostCitiesConfig(n_colors=5, n_ranks=8, hand_size=8)
|
config = LostCitiesConfig(n_colors=5, n_ranks=8, hand_size=8)
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
expeditions = _expeditions(config)
|
expeditions = _expeditions(config)
|
||||||
expeditions[0][0] = [Card(color=0, rank=4)]
|
expeditions[0][0] = [Card(color=0, rank=4)]
|
||||||
expeditions[0][1] = [Card(color=1, rank=4)]
|
expeditions[0][1] = [Card(color=1, rank=4)]
|
||||||
@@ -211,9 +211,9 @@ def test_safe_heuristic_avoids_opening_weak_fifth_color() -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_prefers_followup_on_started_expedition() -> None:
|
def test_heuristic_prefers_followup_on_started_expedition() -> None:
|
||||||
config = LostCitiesConfig(n_colors=3, n_ranks=8, hand_size=5)
|
config = LostCitiesConfig(n_colors=3, n_ranks=8, hand_size=5)
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
expeditions = _expeditions(config)
|
expeditions = _expeditions(config)
|
||||||
expeditions[0][0] = [Card(color=0, rank=4)]
|
expeditions[0][0] = [Card(color=0, rank=4)]
|
||||||
state = make_state(
|
state = make_state(
|
||||||
@@ -233,9 +233,9 @@ def test_safe_heuristic_prefers_followup_on_started_expedition() -> None:
|
|||||||
assert chosen.color == 0
|
assert chosen.color == 0
|
||||||
|
|
||||||
|
|
||||||
def test_safe_heuristic_avoids_unopened_discard_draw_after_four_opens() -> None:
|
def test_heuristic_avoids_unopened_discard_draw_after_four_opens() -> None:
|
||||||
config = LostCitiesConfig(n_colors=5, n_ranks=8, hand_size=8)
|
config = LostCitiesConfig(n_colors=5, n_ranks=8, hand_size=8)
|
||||||
bot = SafeHeuristicBot()
|
bot = HeuristicBot()
|
||||||
expeditions = _expeditions(config)
|
expeditions = _expeditions(config)
|
||||||
expeditions[0][0] = [Card(color=0, rank=4)]
|
expeditions[0][0] = [Card(color=0, rank=4)]
|
||||||
expeditions[0][1] = [Card(color=1, rank=4)]
|
expeditions[0][1] = [Card(color=1, rank=4)]
|
||||||
|
|||||||
@@ -3,13 +3,13 @@ from __future__ import annotations
|
|||||||
from coolrl_lost_cities.games.classic.game import LostCitiesConfig
|
from coolrl_lost_cities.games.classic.game import LostCitiesConfig
|
||||||
|
|
||||||
from coolrl_lost_cities.games.classic.deep_cfr.imitation import (
|
from coolrl_lost_cities.games.classic.deep_cfr.imitation import (
|
||||||
collect_safe_heuristic_samples,
|
collect_heuristic_samples,
|
||||||
new_pretrained_strategy_network,
|
new_pretrained_strategy_network,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_collect_safe_heuristic_samples_shapes() -> None:
|
def test_collect_heuristic_samples_shapes() -> None:
|
||||||
x, y, legal = collect_safe_heuristic_samples(LostCitiesConfig(seed=61), games=1, seed=61)
|
x, y, legal = collect_heuristic_samples(LostCitiesConfig(seed=61), games=1, seed=61)
|
||||||
|
|
||||||
assert len(x) == len(y) == len(legal)
|
assert len(x) == len(y) == len(legal)
|
||||||
assert x.ndim == 2
|
assert x.ndim == 2
|
||||||
|
|||||||
@@ -1494,3 +1494,83 @@ def test_interleaved_regret_matching_no_fallback_unchanged_by_mode() -> None:
|
|||||||
assert fallback_a is False
|
assert fallback_a is False
|
||||||
assert np.allclose(policy_uniform, policy_argmax)
|
assert np.allclose(policy_uniform, policy_argmax)
|
||||||
assert np.allclose(policy_uniform.sum(), 1.0)
|
assert np.allclose(policy_uniform.sum(), 1.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_evaluation_opponents_for_iteration_core_only_when_extended_disabled() -> None:
|
||||||
|
from coolrl_lost_cities.games.classic.deep_cfr.config import EvaluationConfig
|
||||||
|
|
||||||
|
cfg = EvaluationConfig(
|
||||||
|
eval_every=5,
|
||||||
|
opponents=("random", "discard_only", "heuristic_cautious"),
|
||||||
|
extended_eval_every=0,
|
||||||
|
extended_opponents=("heuristic_balanced",),
|
||||||
|
)
|
||||||
|
assert cfg.opponents_for_iteration(0) == ()
|
||||||
|
assert cfg.opponents_for_iteration(3) == ()
|
||||||
|
assert cfg.opponents_for_iteration(5) == (
|
||||||
|
"random",
|
||||||
|
"discard_only",
|
||||||
|
"heuristic_cautious",
|
||||||
|
)
|
||||||
|
assert cfg.opponents_for_iteration(50) == (
|
||||||
|
"random",
|
||||||
|
"discard_only",
|
||||||
|
"heuristic_cautious",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_evaluation_opponents_for_iteration_extends_on_extended_cadence() -> None:
|
||||||
|
from coolrl_lost_cities.games.classic.deep_cfr.config import EvaluationConfig
|
||||||
|
|
||||||
|
cfg = EvaluationConfig(
|
||||||
|
eval_every=5,
|
||||||
|
opponents=("random", "discard_only", "heuristic_cautious"),
|
||||||
|
extended_eval_every=50,
|
||||||
|
extended_opponents=(
|
||||||
|
"heuristic_balanced",
|
||||||
|
"heuristic_aggressive",
|
||||||
|
"heuristic_noisy",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
assert cfg.opponents_for_iteration(5) == (
|
||||||
|
"random",
|
||||||
|
"discard_only",
|
||||||
|
"heuristic_cautious",
|
||||||
|
)
|
||||||
|
assert cfg.opponents_for_iteration(45) == (
|
||||||
|
"random",
|
||||||
|
"discard_only",
|
||||||
|
"heuristic_cautious",
|
||||||
|
)
|
||||||
|
assert cfg.opponents_for_iteration(50) == (
|
||||||
|
"random",
|
||||||
|
"discard_only",
|
||||||
|
"heuristic_cautious",
|
||||||
|
"heuristic_balanced",
|
||||||
|
"heuristic_aggressive",
|
||||||
|
"heuristic_noisy",
|
||||||
|
)
|
||||||
|
assert cfg.opponents_for_iteration(100) == (
|
||||||
|
"random",
|
||||||
|
"discard_only",
|
||||||
|
"heuristic_cautious",
|
||||||
|
"heuristic_balanced",
|
||||||
|
"heuristic_aggressive",
|
||||||
|
"heuristic_noisy",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_evaluation_opponents_for_iteration_dedupes_overlap() -> None:
|
||||||
|
from coolrl_lost_cities.games.classic.deep_cfr.config import EvaluationConfig
|
||||||
|
|
||||||
|
cfg = EvaluationConfig(
|
||||||
|
eval_every=5,
|
||||||
|
opponents=("random", "heuristic_cautious"),
|
||||||
|
extended_eval_every=10,
|
||||||
|
extended_opponents=("heuristic_cautious", "heuristic_balanced"),
|
||||||
|
)
|
||||||
|
assert cfg.opponents_for_iteration(10) == (
|
||||||
|
"random",
|
||||||
|
"heuristic_cautious",
|
||||||
|
"heuristic_balanced",
|
||||||
|
)
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ def test_play_game_for_evaluation_finishes_small_match() -> None:
|
|||||||
|
|
||||||
state, result = play_game_for_evaluation(
|
state, result = play_game_for_evaluation(
|
||||||
build_bot("random", seed=1),
|
build_bot("random", seed=1),
|
||||||
build_bot("passive-discard", seed=2),
|
build_bot("discard-only", seed=2),
|
||||||
config,
|
config,
|
||||||
seed=3,
|
seed=3,
|
||||||
max_steps=200,
|
max_steps=200,
|
||||||
@@ -30,7 +30,7 @@ def test_play_match_alternates_seats_and_reports_rates() -> None:
|
|||||||
|
|
||||||
result = play_match(
|
result = play_match(
|
||||||
make_policy_factory("random"),
|
make_policy_factory("random"),
|
||||||
make_policy_factory("passive-discard"),
|
make_policy_factory("discard-only"),
|
||||||
config,
|
config,
|
||||||
games=4,
|
games=4,
|
||||||
seed=10,
|
seed=10,
|
||||||
@@ -50,7 +50,7 @@ def test_evaluation_cli_smoke_json(capsys) -> None:
|
|||||||
"--bot0",
|
"--bot0",
|
||||||
"random",
|
"random",
|
||||||
"--bot1",
|
"--bot1",
|
||||||
"passive-discard",
|
"discard-only",
|
||||||
"--games",
|
"--games",
|
||||||
"2",
|
"2",
|
||||||
"--seed",
|
"--seed",
|
||||||
|
|||||||
@@ -43,10 +43,10 @@ def test_classic_package_exports_bot_registry_helpers() -> None:
|
|||||||
def test_classic_bot_registry_accepts_reproduction_opponent_names() -> None:
|
def test_classic_bot_registry_accepts_reproduction_opponent_names() -> None:
|
||||||
for name in [
|
for name in [
|
||||||
"random",
|
"random",
|
||||||
"passive_discard",
|
"discard_only",
|
||||||
"safe_heuristic",
|
"heuristic_balanced",
|
||||||
"safe_heuristic_loose",
|
"heuristic_aggressive",
|
||||||
"safe_heuristic_strict",
|
"heuristic_cautious",
|
||||||
"noisy_safe",
|
"heuristic_noisy",
|
||||||
]:
|
]:
|
||||||
assert isinstance(classic.build_bot(name, seed=1), classic.LostCitiesPolicy)
|
assert isinstance(classic.build_bot(name, seed=1), classic.LostCitiesPolicy)
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ def test_gui_argparser_accepts_classic_options() -> None:
|
|||||||
"--mode",
|
"--mode",
|
||||||
"pvc",
|
"pvc",
|
||||||
"--bot",
|
"--bot",
|
||||||
"safe-heuristic",
|
"heuristic-balanced",
|
||||||
"--seed",
|
"--seed",
|
||||||
"7",
|
"7",
|
||||||
"--width",
|
"--width",
|
||||||
@@ -22,7 +22,7 @@ def test_gui_argparser_accepts_classic_options() -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
assert args.mode == "pvc"
|
assert args.mode == "pvc"
|
||||||
assert args.bot == "safe-heuristic"
|
assert args.bot == "heuristic-balanced"
|
||||||
assert args.seed == 7
|
assert args.seed == 7
|
||||||
assert args.width == 1024
|
assert args.width == 1024
|
||||||
assert args.height == 768
|
assert args.height == 768
|
||||||
|
|||||||
@@ -3,19 +3,19 @@ from __future__ import annotations
|
|||||||
import pytest
|
import pytest
|
||||||
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig
|
from coolrl_lost_cities.games.classic.game import GameState, LostCitiesConfig
|
||||||
|
|
||||||
from coolrl_lost_cities.games.classic.bots.heuristic import SafeHeuristicBot
|
from coolrl_lost_cities.games.classic.bots.heuristic import HeuristicBot
|
||||||
from coolrl_lost_cities.games.classic.bots.heuristic_py import (
|
from coolrl_lost_cities.games.classic.bots.heuristic_py import (
|
||||||
SafeHeuristicBot as PythonSafeHeuristicBot,
|
HeuristicBot as PythonHeuristicBot,
|
||||||
)
|
)
|
||||||
from coolrl_lost_cities.games.classic.bots.registry import (
|
from coolrl_lost_cities.games.classic.bots.registry import (
|
||||||
LOOSE_SAFE_HEURISTIC_PARAMS,
|
AGGRESSIVE_HEURISTIC_PARAMS,
|
||||||
STRICT_SAFE_HEURISTIC_PARAMS,
|
CAUTIOUS_HEURISTIC_PARAMS,
|
||||||
)
|
)
|
||||||
|
|
||||||
VARIANTS = (
|
VARIANTS = (
|
||||||
("default", None),
|
("default", None),
|
||||||
("loose", LOOSE_SAFE_HEURISTIC_PARAMS),
|
("loose", AGGRESSIVE_HEURISTIC_PARAMS),
|
||||||
("strict", STRICT_SAFE_HEURISTIC_PARAMS),
|
("strict", CAUTIOUS_HEURISTIC_PARAMS),
|
||||||
)
|
)
|
||||||
|
|
||||||
CONFIGS = (
|
CONFIGS = (
|
||||||
@@ -28,14 +28,14 @@ CONFIGS = (
|
|||||||
@pytest.mark.parametrize(("variant_name", "params"), VARIANTS)
|
@pytest.mark.parametrize(("variant_name", "params"), VARIANTS)
|
||||||
@pytest.mark.parametrize("config", CONFIGS)
|
@pytest.mark.parametrize("config", CONFIGS)
|
||||||
@pytest.mark.parametrize("seed", range(2))
|
@pytest.mark.parametrize("seed", range(2))
|
||||||
def test_cython_safe_heuristic_matches_python_action_sequence(
|
def test_cython_heuristic_matches_python_action_sequence(
|
||||||
variant_name: str,
|
variant_name: str,
|
||||||
params,
|
params,
|
||||||
config: LostCitiesConfig,
|
config: LostCitiesConfig,
|
||||||
seed: int,
|
seed: int,
|
||||||
) -> None:
|
) -> None:
|
||||||
py_bot = PythonSafeHeuristicBot(params)
|
py_bot = PythonHeuristicBot(params)
|
||||||
cy_bot = SafeHeuristicBot(params)
|
cy_bot = HeuristicBot(params)
|
||||||
py_state = GameState.new_game(config, seed=seed)
|
py_state = GameState.new_game(config, seed=seed)
|
||||||
cy_state = GameState.new_game(config, seed=seed)
|
cy_state = GameState.new_game(config, seed=seed)
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user