Archive docs markdown files

2026-05-07 15:39:24 +09:00
parent 2414b651d5
commit 2c96c5ee82
14 changed files with 0 additions and 0 deletions
@@ -0,0 +1,82 @@
+# Classic Port Notes
+
+This repository starts as a focused extraction of the Lost Cities game from the
+legacy `coolrl` repository. The first target is the classic two-player card
+game, without the earlier training-oriented tiers.
+
+## Current Direction
+
+- Implement the classic Lost Cities rules first.
+- Treat classic as the initial concrete game under `coolrl_lost_cities.games`.
+- Do not carry over `tier0` through `tier3`; those were useful for experiments,
+  but they should not shape the first public game API.
+- Use the in-process Python/Cython implementation for the current port.
+- Keep the GUI in scope for local play, but do not carry a separate native backend.
+- Keep RL and training code out of the first extraction.
+
+The expected package shape is roughly:
+
+```text
+src/coolrl_lost_cities/
+  games/
+    classic/
+      game.pyx
+      snapshots.py
+      evaluation.py
+      env.py
+      policy.py
+      bots/
+      pygame_pvp.py
+      fixtures/
+      assets/
+      docs/
+```
+
+Tests should live outside the package, roughly under:
+
+```text
+tests/games/classic/
+```
+
+## Out Of Scope For The First Port
+
+- Deep CFR
+- General training infrastructure
+- Evaluation loops for learned policies
+- Separate native backend support
+- Web client
+- Legacy experiment configs, checkpoints, logs, exports, and analysis artifacts
+
+Bot-vs-bot helpers can stay with the classic game if they are useful for smoke
+tests and local play. Broader policy evaluation can be introduced later with the
+training layer.
+
+## Later Training Shape
+
+If training is added later, it should not make Deep CFR the center of the
+package. Evaluation and policy interfaces should be general enough for multiple
+approaches, with Deep CFR as one implementation.
+
+A possible future shape:
+
+```text
+src/coolrl_lost_cities/
+  games/
+    classic/
+  training/
+    policies.py
+    evaluation.py
+    deep_cfr/
+    imitation/
+    policy_gradient/
+```
+
+The game package should expose rules, state transitions, legal actions, scoring,
+and playable UI. Training code can adapt those pieces later.
+
+## Naming Notes
+
+For now, use `classic` for the five-expedition game. Other variants, such as a
+six-expedition version, can be added later if needed. The current port should
+avoid adding a variant registry or broad abstraction before there is a second
+concrete game to support.
@@ -0,0 +1,111 @@
+# Deep CFR Batched Evaluation 2026-05-07
+
+Run directory:
+
+`/mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_032944_deep_cfr_batched_eval_cuda_postprocess_1iter`
+
+Command:
+
+```bash
+uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli train \
+  --config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
+  --checkpoint-dir /mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_032944_deep_cfr_batched_eval_cuda_postprocess_1iter \
+  --max-iterations 1 \
+  --eval-every 1 \
+  --save-latest-only
+```
+
+The config used `evaluation.batch_size: 64` and `evaluation.device: trainer`.
+The base run device was CUDA.
+
+## Summary
+
+The run completed one training iteration and evaluated immediately.
+
+| Metric | Value |
+| --- | ---: |
+| `iteration_seconds` | 21.322356 |
+| `evaluation_seconds` | 14.834096 |
+| `traversal_seconds` | 3.850308 |
+| `advantage_train_seconds` | 1.778991 |
+| `strategy_train_seconds` | 0.847232 |
+
+Compared with the previous batch-size-1 CUDA check, evaluation time changed
+from about 61.826s to about 14.834s.
+
+## Opponent Timing
+
+| Opponent | elapsed | avg len | policy turns | network | postprocess | encoding | legal mask | opponent act |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| `safe_heuristic_strict` | 3.971 | 1000.0 | 50000 | 0.195 | 0.251 | 0.147 | 0.084 | 3.168 |
+| `safe_heuristic` | 3.685 | 997.5 | 49876 | 0.199 | 0.257 | 0.148 | 0.084 | 2.868 |
+| `safe_heuristic_loose` | 3.330 | 980.8 | 49038 | 0.194 | 0.250 | 0.145 | 0.082 | 2.536 |
+| `noisy_safe` | 2.902 | 955.9 | 47784 | 0.195 | 0.250 | 0.144 | 0.079 | 2.109 |
+| `random` | 0.797 | 640.4 | 31968 | 0.154 | 0.204 | 0.095 | 0.053 | 0.212 |
+| `passive_discard` | 0.138 | 172.2 | 8562 | 0.032 | 0.042 | 0.025 | 0.014 | 0.004 |
+
+## Totals
+
+| Metric | Value |
+| --- | ---: |
+| `eval_elapsed_seconds` | 14.824448 |
+| `eval_policy_turns` | 237228 |
+| `eval_policy_network_seconds` | 0.968006 |
+| `eval_policy_postprocess_seconds` | 1.253108 |
+| `eval_policy_encoding_seconds` | 0.702534 |
+| `eval_policy_legal_mask_seconds` | 0.396494 |
+| `eval_opponent_act_seconds` | 10.896567 |
+| `eval_apply_action_seconds` | 0.068075 |
+| `eval_diagnostics_seconds` | 0.060940 |
+
+## Notes
+
+Batched network inference removed the previous batch-size-1 CUDA network
+bottleneck. After batching, safe heuristic opponent action time became the
+largest remaining eval cost for safe heuristic opponents.
+
+The first batched implementation exposed a postprocess synchronization cost
+from per-row entropy calculation. Moving entropy calculation into torch batch
+postprocess reduced that cost before this run.
+
+## Opponent Parallel Evaluation
+
+Run directory:
+
+`/mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_033925_deep_cfr_batched_eval_parallel_1iter`
+
+Command:
+
+```bash
+uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli train \
+  --config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
+  --checkpoint-dir /mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_033925_deep_cfr_batched_eval_parallel_1iter \
+  --max-iterations 1 \
+  --eval-every 1 \
+  --save-latest-only
+```
+
+This run used `evaluation.num_workers: 4`, `evaluation.batch_size: 64`, and
+`evaluation.device: trainer` on CUDA.
+
+| Metric | Batched sequential | Batched opponent-parallel |
+| --- | ---: | ---: |
+| `iteration_seconds` | 21.322356 | 12.854383 |
+| `evaluation_seconds` | 14.834096 | 6.420402 |
+| `traversal_seconds` | 3.850308 | 3.852627 |
+| `advantage_train_seconds` | 1.778991 | 1.746623 |
+| `strategy_train_seconds` | 0.847232 | 0.819998 |
+
+Opponent elapsed values from the parallel run:
+
+| Opponent | elapsed | opponent act | network | avg len |
+| --- | ---: | ---: | ---: | ---: |
+| `safe_heuristic_strict` | 4.121 | 3.225 | 0.196 | 1000.0 |
+| `safe_heuristic` | 4.029 | 2.863 | 0.281 | 997.5 |
+| `safe_heuristic_loose` | 3.807 | 2.631 | 0.286 | 981.6 |
+| `noisy_safe` | 3.026 | 2.141 | 0.211 | 951.7 |
+| `random` | 1.130 | 0.212 | 0.253 | 639.0 |
+| `passive_discard` | 0.417 | 0.004 | 0.114 | 172.2 |
+
+Parallel eval reduced measured eval wall time from about 14.83s to about
+6.42s, roughly `2.31x` faster for this one-iteration profile.
@@ -0,0 +1,153 @@
+# Deep CFR Evaluation Profile 2026-05-07
+
+Run directory:
+
+`/mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_030634_deep_cfr_profile_eval_breakdown_10iter`
+
+Command:
+
+```bash
+uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli train \
+  --config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
+  --checkpoint-dir /mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_030634_deep_cfr_profile_eval_breakdown_10iter \
+  --max-iterations 10 \
+  --save-latest-only
+```
+
+## Summary
+
+The run completed 10 iterations. Loss values stayed finite.
+
+Evaluation ran on iterations 5 and 10.
+
+| Iteration | `iteration_seconds` | `evaluation_seconds` |
+| ---: | ---: | ---: |
+| 5 | 16.473433 | 11.055458 |
+| 10 | 16.835262 | 10.855797 |
+
+## Opponent Averages
+
+Values below are averaged across iterations 5 and 10.
+
+| Opponent | elapsed | avg len | policy select | network | postprocess | encoding | legal mask | opponent act |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| `safe_heuristic_strict` | 2.184393 | 158.13 | 1.628546 | 1.279717 | 0.175923 | 0.043837 | 0.038600 | 0.518953 |
+| `random` | 2.072657 | 186.11 | 1.918007 | 1.515231 | 0.207319 | 0.051491 | 0.037316 | 0.112155 |
+| `safe_heuristic` | 1.942907 | 143.23 | 1.471839 | 1.155444 | 0.159592 | 0.039683 | 0.035180 | 0.435870 |
+| `noisy_safe` | 1.904932 | 143.71 | 1.493521 | 1.174909 | 0.161256 | 0.039845 | 0.034331 | 0.374162 |
+| `safe_heuristic_loose` | 1.829347 | 136.13 | 1.400569 | 1.099200 | 0.151752 | 0.037722 | 0.033497 | 0.394891 |
+| `passive_discard` | 1.014156 | 96.82 | 0.982162 | 0.774097 | 0.105579 | 0.026702 | 0.021929 | 0.006383 |
+
+Other averaged step costs were small:
+
+| Opponent | apply action | diagnostics | final scoring |
+| --- | ---: | ---: | ---: |
+| `safe_heuristic_strict` | 0.005185 | 0.008161 | 0.000569 |
+| `random` | 0.006229 | 0.009001 | 0.000632 |
+| `safe_heuristic` | 0.004748 | 0.007636 | 0.000581 |
+| `noisy_safe` | 0.004818 | 0.007761 | 0.000584 |
+| `safe_heuristic_loose` | 0.004473 | 0.007438 | 0.000592 |
+| `passive_discard` | 0.002911 | 0.005156 | 0.000477 |
+
+## Notes
+
+`policy_select_seconds` dominated every opponent.
+
+Inside policy selection, `policy_network_seconds` was the largest component.
+`policy_postprocess_seconds` was second. `policy_encoding_seconds` and
+`policy_legal_mask_seconds` were much smaller.
+
+`opponent_act_seconds` was meaningful for safe heuristic opponents, but was
+still smaller than policy network time.
+
+`apply_action_seconds`, `diagnostics_seconds`, and `final_scoring_seconds` were
+small in this run.
+
+## CPU vs CUDA Evaluation Check
+
+This check compared `--device cpu` and `--device cuda` on the same base
+configuration after the evaluation breakdown metrics were available. The base
+configuration was:
+
+`configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml`
+
+Both runs used one training iteration and ran evaluation on that iteration.
+The base configuration's evaluation settings were kept at 100 games per
+opponent and the six configured opponents.
+
+CPU run directory:
+
+`/mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_eval_device_cpu_1iter`
+
+CPU command:
+
+```bash
+uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli train \
+  --config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
+  --checkpoint-dir /mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_eval_device_cpu_1iter \
+  --max-iterations 1 \
+  --eval-every 1 \
+  --save-latest-only \
+  --device cpu
+```
+
+CUDA run directory:
+
+`/mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_eval_device_cuda_1iter`
+
+CUDA command:
+
+```bash
+uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli train \
+  --config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
+  --checkpoint-dir /mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_eval_device_cuda_1iter \
+  --max-iterations 1 \
+  --eval-every 1 \
+  --save-latest-only \
+  --device cuda
+```
+
+Top-level timing:
+
+| Metric | CPU | CUDA | CUDA / CPU |
+| --- | ---: | ---: | ---: |
+| `iteration_seconds` | 50.969 | 72.501 | 1.42 |
+| `evaluation_seconds` | 38.920 | 61.826 | 1.59 |
+| `traversal_seconds` | 6.343 | 6.409 | 1.01 |
+| `advantage_train_seconds` | 4.054 | 2.884 | 0.71 |
+| `strategy_train_seconds` | 1.643 | 1.369 | 0.83 |
+
+Opponent elapsed timing:
+
+| Opponent | CPU elapsed | CUDA elapsed | CUDA / CPU |
+| --- | ---: | ---: | ---: |
+| `random` | 4.038 | 7.220 | 1.79 |
+| `passive_discard` | 0.990 | 1.773 | 1.79 |
+| `safe_heuristic` | 8.701 | 13.518 | 1.55 |
+| `safe_heuristic_loose` | 8.334 | 13.145 | 1.58 |
+| `safe_heuristic_strict` | 9.097 | 13.882 | 1.53 |
+| `noisy_safe` | 7.751 | 12.281 | 1.58 |
+
+Opponent breakdown for the CPU run:
+
+| Opponent | elapsed | policy turns | network | network / turn | postprocess | encoding | legal mask | opponent act | avg len |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| `random` | 4.038 | 32170 | 2.401 | 0.075 ms | 0.586 | 0.169 | 0.114 | 0.361 | 644.4 |
+| `passive_discard` | 0.990 | 8562 | 0.633 | 0.074 ms | 0.155 | 0.044 | 0.034 | 0.010 | 172.2 |
+| `safe_heuristic` | 8.701 | 49756 | 3.650 | 0.073 ms | 0.903 | 0.253 | 0.211 | 3.068 | 995.1 |
+| `safe_heuristic_loose` | 8.334 | 48958 | 3.592 | 0.073 ms | 0.886 | 0.248 | 0.207 | 2.798 | 979.2 |
+| `safe_heuristic_strict` | 9.097 | 50000 | 3.678 | 0.074 ms | 0.910 | 0.254 | 0.214 | 3.420 | 1000.0 |
+| `noisy_safe` | 7.751 | 47276 | 3.501 | 0.074 ms | 0.864 | 0.244 | 0.195 | 2.354 | 945.7 |
+| Total | 38.911 | 236722 | 17.456 | 0.074 ms | 4.304 | 1.212 | 0.975 | 12.011 |  |
+
+Opponent breakdown for the CUDA run:
+
+| Opponent | elapsed | policy turns | network | network / turn | postprocess | encoding | legal mask | opponent act | avg len |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| `random` | 7.220 | 32096 | 5.337 | 0.166 ms | 0.716 | 0.179 | 0.129 | 0.384 | 642.9 |
+| `passive_discard` | 1.773 | 8562 | 1.364 | 0.159 ms | 0.185 | 0.047 | 0.039 | 0.012 | 172.2 |
+| `safe_heuristic` | 13.518 | 49876 | 8.053 | 0.161 ms | 1.097 | 0.274 | 0.239 | 3.139 | 997.5 |
+| `safe_heuristic_loose` | 13.145 | 49078 | 7.967 | 0.162 ms | 1.080 | 0.271 | 0.236 | 2.890 | 981.6 |
+| `safe_heuristic_strict` | 13.882 | 50000 | 8.034 | 0.161 ms | 1.096 | 0.274 | 0.239 | 3.524 | 1000.0 |
+| `noisy_safe` | 12.281 | 47266 | 7.650 | 0.162 ms | 1.044 | 0.262 | 0.219 | 2.421 | 945.5 |
+| Total | 61.819 | 236878 | 38.406 | 0.162 ms | 5.218 | 1.307 | 1.101 | 12.370 |  |
@@ -0,0 +1,50 @@
+# Deep CFR Evaluation Profile Plan
+
+Goal: split evaluation runtime into the main per-step costs so eval iteration
+spikes can be explained without guessing.
+
+The profiling run should keep the normal full config and `eval_every: 5`, then
+compare iterations 5 and 10.
+
+## Metrics
+
+Each metric is emitted with the existing opponent prefix:
+`eval_<opponent>_<metric_name>`.
+
+Top-level counters:
+
+- `policy_turns`
+- `opponent_turns`
+- `avg_game_length`
+- `elapsed_seconds`
+- `games_per_second`
+- `steps_per_second`
+
+Step-level runtime:
+
+- `policy_select_seconds`
+- `opponent_act_seconds`
+- `apply_action_seconds`
+- `diagnostics_seconds`
+- `final_scoring_seconds`
+
+Policy select breakdown:
+
+- `policy_legal_mask_seconds`
+- `policy_encoding_seconds`
+- `policy_network_seconds`
+- `policy_postprocess_seconds`
+
+## Interpretation
+
+If `policy_network_seconds` dominates, evaluation is mostly batch-size-1 model
+inference overhead.
+
+If `policy_encoding_seconds` or `policy_legal_mask_seconds` dominates, the eval
+policy path is paying per-turn state feature/mask construction costs.
+
+If `opponent_act_seconds` dominates, the opponent bot implementation is the
+main eval cost for that opponent.
+
+If `apply_action_seconds` dominates, the game step path itself is the eval
+bottleneck.
@@ -0,0 +1,213 @@
+# Deep CFR Legacy Experiment Reproduction Plan
+
+This document tracks what is needed to reproduce the legacy `../coolrl`
+experiment below with the same intended semantics in this repository:
+
+`experiments/lost_cities/deep_cfr_pure_self_play_zero_pit_poc_full_depth_slot_aware_playability`
+
+The goal is not to run a similar Deep CFR configuration. The goal is to map the
+legacy experiment's hyperparameters and feature semantics into this repository's
+own config schema so that the training run means the same thing.
+
+Exact legacy YAML compatibility is not required. It is acceptable to create a
+new config file under this repository as long as every relevant legacy
+hyperparameter is represented explicitly and the differences are documented.
+
+Mapped config in this repository:
+
+`configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml`
+
+## Source Experiment
+
+Legacy config path:
+
+`/home/coolguy/dev/coolrl/experiments/lost_cities/deep_cfr_pure_self_play_zero_pit_poc_full_depth_slot_aware_playability/config.yaml`
+
+Important legacy settings:
+
+1. `seed: 79`
+2. `max_hours: 4`
+3. `max_iterations: null`
+4. `device: CUDA`
+5. `use_amp: false`
+6. `rules.tier: tier3`
+7. `encoding.derived_playability: true`
+8. `encoding.slot_aware_playability: true`
+9. `network.hidden_size: 256`
+10. `network.num_layers: 3`
+11. `network.activation: relu`
+12. `traversal.traversals_per_player: 70`
+13. `traversal.max_depth: null`
+14. `traversal.max_nodes_per_traversal: 1000`
+15. `traversal.opponent_policy: self_play_league`
+16. `traversal.outcome_sampling_epsilon: 0.2`
+17. `traversal.outcome_sampling_value_clip: 500`
+18. `traversal.outcome_unsampled_regret: zero`
+19. `optimization.advantage_batch_size: 1024`
+20. `optimization.strategy_batch_size: 1024`
+21. `optimization.advantage_updates_per_iteration: 256`
+22. `optimization.strategy_updates_per_iteration: 256`
+23. `optimization.learning_rate: 3.0e-5`
+24. `optimization.weight_decay: 1.0e-4`
+25. `optimization.grad_clip: 1.0`
+
+## Required Reproduction Work
+
+### Config
+
+Represent the legacy hyperparameters in this repository's config schema:
+
+1. Top-level:
+   - `experiment_name`
+   - `seed`
+   - `max_iterations`
+   - `max_hours`
+   - `device`
+   - `use_amp`
+2. `rules`:
+   - `tier`
+   - optional direct `LostCitiesConfig` field overrides
+3. `encoding`:
+   - `derived_playability`
+   - `slot_aware_playability`
+4. `network`:
+   - `hidden_size`
+   - `num_layers`
+   - `activation`
+5. `traversal` semantics:
+   - per-player traversal count
+   - max nodes per traversal
+   - traversal worker chunk size
+   - self-play league settings
+6. `optimization` semantics:
+   - separate advantage and strategy batch sizes
+   - separate advantage and strategy update counts
+   - `weight_decay`
+   - `grad_clip`
+7. `evaluation.on_max_steps`
+8. `checkpoint`:
+   - `save_iteration_interval`
+   - `save_latest_only`
+   - `progress_interval_seconds`
+
+### Encoding
+
+Port the legacy feature semantics:
+
+1. Base information-state encoding must remain deterministic.
+2. Add `derived_playability` color-level features.
+3. Add `slot_aware_playability` hand-slot action-local features.
+4. Preserve feature order and dimensions from legacy where possible.
+5. Add tests for input dimension and known-state feature values.
+
+The slot-aware block is the core of the source experiment. Without this block,
+the reproduction is not meaningful.
+
+### Network
+
+Make the MLP configurable:
+
+1. `hidden_size`
+2. `num_layers`
+3. `activation`
+
+The legacy experiment uses a 3-layer ReLU MLP with hidden size 256.
+
+### Optimization
+
+Match the legacy training knobs:
+
+1. Separate advantage and strategy batch sizes.
+2. Separate advantage and strategy update counts per iteration.
+3. Adam `weight_decay`.
+4. Gradient clipping.
+
+### Traversal
+
+Match legacy traversal semantics and metrics:
+
+1. Per-player traversal count behavior.
+2. Full-depth traversal when `max_depth: null`.
+3. Max nodes per traversal.
+4. Self-play league behavior.
+5. Endpoint depth accounting:
+   - endpoint depth sum
+   - endpoint depth buckets
+   - depth bucket width
+   - depth bucket max
+6. Optional worker progress logging.
+7. Optional hotspot profiling.
+
+### Run Loop And Checkpointing
+
+Match legacy runtime behavior:
+
+1. Stop by `max_hours`.
+2. Stop by `max_iterations` when set.
+3. Allow `max_iterations: null`.
+4. Save every N iterations through `save_iteration_interval`.
+5. Support `save_latest_only`.
+6. Preserve useful config artifacts in the run directory.
+7. Keep `metrics.jsonl`, `runtime_progress.json`, and `train.log` useful for
+   comparing this run against the legacy report. Exact artifact layout
+   compatibility is optional.
+
+### Evaluation And Metrics
+
+Match the legacy evaluation configuration:
+
+1. `evaluation.on_max_steps`.
+2. Opponent list:
+   - `random`
+   - `passive_discard`
+   - `safe_heuristic`
+   - `safe_heuristic_loose`
+   - `safe_heuristic_strict`
+   - `noisy_safe`
+3. Metric suffixes used by the legacy analysis scripts, including opening
+   quality, opened-color distribution, discard take rates, expedition quality,
+   policy entropy, and timeout counts.
+4. Preserve comparable semantics for the legacy experiment's decision criteria.
+5. Exact metric key/schema compatibility is optional unless legacy `analyze.py`
+   is reused directly.
+6. Confirm whether the current classic evaluator emits every required core
+   metric.
+7. Add a small adapter or mapping table if this repository keeps different
+   metric names.
+
+### Analysis Compatibility
+
+The legacy `analyze.py` expects specific run artifacts and metric keys. Direct
+reuse is optional. The required goal is that the generated run data can be
+compared against the legacy report through documented metric semantics:
+
+1. Core metric meanings are documented.
+2. Latest iteration and latest eval iteration can be inferred.
+3. Endpoint-depth metrics are present or explicitly marked as omitted.
+4. Evaluation metric prefixes and suffixes are documented when they differ from
+   legacy output.
+5. If we choose to reuse legacy `analyze.py` directly, then add a compatibility
+   adapter for keys and artifact layout.
+
+## Suggested Commit Breakdown
+
+1. Reproduction config schema and mapped experiment preset.
+2. Configurable network and optimizer knobs.
+3. Derived and slot-aware encoding.
+4. Run loop, checkpoint, and evaluation compatibility.
+5. Traversal endpoint metrics, progress, and profile compatibility.
+6. Metric semantics and optional analysis adapter pass.
+
+## Definition Of Done
+
+The reproduction work is complete when:
+
+1. A mapped config exists in this repository for the legacy experiment.
+2. Every relevant legacy hyperparameter is either represented or explicitly
+   marked as intentionally irrelevant.
+3. A smoke-sized version of the mapped config runs end to end.
+4. The full mapped config starts training with the same key semantics.
+5. The information-state input dimension matches the legacy slot-aware setup for
+   tier3.
+6. The core training metrics and evaluation metrics are semantically comparable
+   with the legacy experiment report.
@@ -0,0 +1,99 @@
+# Deep CFR Legacy Runtime Comparison 2026-05-07
+
+This note records a runtime summary from the older `../coolrl` Lost Cities
+Deep CFR implementation and compares it with the current profiling runs in
+this repository.
+
+## Legacy Run Summary
+
+The older run completed metrics through iteration 387. The process had stopped
+before completing iteration 388.
+
+Total elapsed time through iteration 387:
+
+`5879.17s`, or about `1h 37m 59s`.
+
+| Segment | Mean | Median | Note |
+| --- | ---: | ---: | --- |
+| All iterations | 15.19s/iter | 11.44s | Includes eval iterations |
+| Non-eval iterations | 11.51s/iter | 11.32s | Normal training iteration |
+| Eval iterations | 30.00s/iter | 29.31s | Eval every 5 iterations |
+| Evaluation only | 18.61s/eval | 18.04s | Early evals were slower |
+| Traversal | 7.16s/iter | 6.98s | 140 traversals/iter |
+| Advantage train | 2.81s/iter | 2.78s | Player 0 + player 1 |
+| Strategy train | 1.44s/iter | 1.44s | |
+| Overall throughput | 5387 nodes/s | 5437 nodes/s | |
+| Traversal throughput | 19.8 traversals/s | 20.1 traversals/s | |
+
+Recent 50 iteration window from that run:
+
+| Segment | Mean |
+| --- | ---: |
+| All iterations | 15.57s/iter |
+| Non-eval iterations | 12.21s/iter |
+| Recent 20 evals, eval only | 17.31s/eval |
+| Recent 20 eval iterations | 29.30s/iter |
+
+Evaluation ran every 5 iterations: 5, 10, 15, ..., 385.
+
+The practical legacy cadence was roughly:
+
+`4 normal iterations + 1 eval iteration ~= 75s per 5 iterations`.
+
+## Current Repo Reference Points
+
+From `docs/deep-cfr-profile-advantage-memory-split-2026-05-07.md`:
+
+| Segment | Current mean |
+| --- | ---: |
+| Non-eval iterations | 5.832958s/iter |
+| Traversal | 3.160975s/iter |
+| Advantage train | 1.742895s/iter |
+| Strategy train | 0.912324s/iter |
+
+From `docs/deep-cfr-batched-evaluation-2026-05-07.md`:
+
+| Segment | Current value |
+| --- | ---: |
+| Batched CUDA evaluation | 14.834096s/eval |
+| Batched CUDA 1-iter wall time with eval | 21.322356s |
+| Batched opponent-parallel CUDA evaluation | 6.420402s/eval |
+| Batched opponent-parallel CUDA 1-iter wall time with eval | 12.854383s |
+
+## Rough Comparison
+
+Normal training iterations improved from about `11.51s` to about `5.83s`,
+roughly `1.97x` faster.
+
+Traversal improved from about `7.16s` to about `3.16s`, roughly `2.27x`
+faster.
+
+Evaluation improved from about `18.61s` to about `14.83s`, roughly `1.25x`
+faster for the measured batched CUDA profile. With opponent-parallel eval, the
+measured eval time was about `6.42s`, roughly `2.90x` faster than the legacy
+eval-only average.
+
+Using the simple cadence model:
+
+Legacy:
+
+`4 * 11.51 + (11.51 + 18.61) = 76.16s per 5 iterations`
+
+Current batched sequential:
+
+`4 * 5.83 + (5.83 + 14.83) = 43.98s per 5 iterations`
+
+Current batched opponent-parallel:
+
+`4 * 5.83 + (5.83 + 6.42) = 35.57s per 5 iterations`
+
+That implies about `1.73x` faster eval-included wall time for the batched
+sequential rough comparison, and about `2.14x` faster for the batched
+opponent-parallel rough comparison.
+
+## Caveat
+
+The legacy numbers came from a long run through iteration 387. The current
+numbers are from targeted profiling runs. The comparison is useful for order of
+magnitude and bottleneck direction, not as a strict benchmark under identical
+runtime conditions.
@@ -0,0 +1,263 @@
+# Deep CFR `opponent_policy: network` 발산 (Policy Collapse) 분석
+
+**Date:** 2026-05-07
+**Author:** Claude Code 세션 기록
+**Status:** Definitive — `opponent_policy: network`는 안정적인 학습을 보장하지 않음
+
+## TL;DR
+
+`traversal.opponent_policy: network` 옵션은 **자기 자신의 현재 네트워크**를
+opponent로 사용한다. 이 설정으로 1000-iteration 실험을 두 번 (512x3 / 1024x4)
+돌린 결과, **두 실험 모두 학습 초기 (peak) 이후 명백한 policy collapse가
+발생**했다. 큰 네트워크는 발산을 늦추기는 하지만 막지는 못한다.
+
+**권고:** 향후 학습은 `opponent_policy: self_play_league`를 기본값으로 유지할 것.
+
+## 실험 설정
+
+두 실험 모두 동일한 구조:
+
+| 항목 | 값 |
+|------|-----|
+| `traversal.opponent_policy` | `network` |
+| `traversal.traversals_per_iteration` | 2 |
+| `traversal.traversals_per_player` | 70 |
+| `optimization.advantage_updates_per_iteration` | 512 |
+| `optimization.strategy_updates_per_iteration` | 512 |
+| `optimization.learning_rate` | 3e-5 |
+| `regret_matching.all_negative_fallback` | argmax_tiebreak |
+| `self_play.max_snapshots` | 0 (snapshot pool 비활성) |
+| Eval | 매 5 iter, 6 opponents × 100 games |
+| Max iterations | 1000 |
+
+차이점:
+- **512x3**: `network.hidden_size=512, num_layers=3`
+  config: `configs/deep_cfr/deep_cfr_opponent_network_512x3_1000iter.yaml`
+- **1024x4**: `network.hidden_size=1024, num_layers=4`
+  config: `configs/deep_cfr/deep_cfr_opponent_network_1024x4_1000iter.yaml`
+
+두 실험 모두 동일 머신 (AMD Ryzen 5 5600X, 12 threads, CUDA GPU)에서
+약간의 시간차 (4분)를 두고 병렬 실행. 병렬 실행으로 iteration time이
+평균 ~2배 느려졌으나 결과 패턴(발산)에는 영향 없음.
+
+## 결과
+
+### 512x3 (iteration 363까지 관찰 후 중단)
+
+| Phase | Iter | Random WR | Random Δ | Safe Heur WR | Safe Heur Δ |
+|-------|------|-----------|----------|---------------|-------------|
+| 초기 | 5 | 53.0% | -0.7 | 4.0% | -78.1 |
+| **Peak** | **15** | **85.0%** | **+32.2** | 5.0% | -47.2 |
+| 발산 시작 | 20 | 67.0% | +14.4 | 2.0% | -62.3 |
+| 발산 진행 | 30 | 36.0% | -14.3 | 3.0% | -106.2 |
+| 수렴 (망함) | 100 | 41.0% | -12.9 | 0.0% | -111.6 |
+| 수렴 (망함) | 200 | 33.0% | -17.1 | 1.0% | -114.1 |
+| 마지막 | 360 | 34.0% | -12.7 | 2.0% | -101.3 |
+
+- **Peak는 iter 15** — Random 상대 85% win rate.
+- iter 20부터 급격히 무너짐.
+- iter 30 이후 Random WR이 **Random 자신(50%)보다 낮음** → 모델이
+  random보다 못 한 상태로 수렴.
+- Safe Heuristic 상대로는 처음부터 끝까지 ~0–8% (전혀 학습 안 됨).
+
+### 1024x4 (iteration 231까지 관찰 후 중단)
+
+| Phase | Iter | Random WR | Random Δ | Safe Heur WR | Safe Heur Δ |
+|-------|------|-----------|----------|---------------|-------------|
+| 초기 | 5 | 35.0% | -14.9 | 1.0% | -91.8 |
+| Plateau | 30 | 62.0% | +15.0 | 3.0% | -82.2 |
+| Plateau | 55 | 63.0% | +16.0 | 5.0% | -69.2 |
+| **Best Random** | **70** | **70.0%** | **+17.6** | 6.0% | -74.1 |
+| **Best Safe Heur** | **85** | 63.0% | +11.4 | **13.0%** | **-52.3** |
+| 발산 시작 | 100 | 49.0% | +3.8 | 3.0% | -89.7 |
+| 발산 진행 | 150 | 35.0% | -17.4 | 5.0% | -78.1 |
+| 수렴 (망함) | 200 | 24.0% | -24.6 | 9.0% | -71.3 |
+| 마지막 | 230 | 36.0% | -19.9 | 4.0% | -95.6 |
+
+- Plateau가 **iter 30–95** (60+ iter)로 길게 유지됨.
+- **Best Safe Heuristic WR = 13% (iter 85)** — 512x3의 best (8%, iter 10)
+  보다 명확히 우수. 큰 capacity가 다양성 표현에 도움.
+- iter 100 이후부터 발산 시작, iter 150 이후 망가짐.
+
+### 두 실험 비교
+
+| 항목 | 512x3 | 1024x4 |
+|------|-------|--------|
+| Best Random WR | **85%** (iter 15) | 70% (iter 70) |
+| Best Safe Heur WR | 8% (iter 10) | **13%** (iter 85) |
+| Plateau 길이 | ~10 iter | ~65 iter |
+| 발산 시작 시점 | iter 20 | iter 100 |
+| 수렴 시 Random WR | ~30–40% | ~30–40% |
+
+**관찰:**
+- 큰 네트워크는 **plateau를 5x 이상 길게** 유지함.
+- 큰 네트워크는 Safe Heuristic 상대로도 plateau 구간에 학습 가능 (13%).
+- 그러나 **두 실험 모두 결국 발산**. capacity로 발산을 막지 못함.
+
+## 원인 분석
+
+### 1. Moving Target (가장 큰 원인)
+
+정상적인 외부 샘플링 CFR은 traversal 동안 opponent가 **고정된** policy를
+사용한다고 가정한다. `opponent_policy: network`는 **학습 중인 네트워크**를
+opponent로 사용하기 때문에:
+
+- 매 iteration마다 opponent의 policy가 바뀜.
+- 이전 iteration에서 추정한 advantage가 outdated 됨.
+- advantage memory의 (state, action, regret) 샘플들이 서로 다른 opponent
+  policy 하에서 측정됨 → 일관성 없음.
+- 결과: 학습이 자기 자신을 쫓는 무한 루프.
+
+### 2. Echo Chamber (다양성 부재)
+
+traverser와 opponent가 **같은 네트워크**를 공유하므로:
+
+- 같은 약점 공유. 예를 들어 모델이 Safe Heuristic 스타일의 조심스러운
+  상대를 처리하지 못하면, 그 상대를 self-play에서 만날 일이 없음.
+- 약점이 advantage estimation에 표현되지 않음 → regret 신호로 학습되지
+  않음.
+- 결과: 모델이 "자기 자신을 이기는 데 특화된" 좁은 strategy로 수렴.
+  Random 같은 다른 distribution을 만나면 처참히 패배.
+
+### 3. CFR 수렴 보장 깨짐
+
+Deep CFR의 이론적 수렴은 **average strategy**가 Nash에 가까워진다는
+보장이며, opponent가 fixed 또는 average policy일 때 성립한다. network
+policy를 매번 바뀌는 traversal opponent로 사용하면:
+
+- external sampling의 unbiased estimate 가정 위반.
+- no-regret 보장 사라짐.
+- 수렴이 이론적으로 보장되지 않음 → 실제로 발산 관찰됨.
+
+### 4. Non-stationary Regret 추정
+
+같은 (state, action) 샘플에서 측정한 regret이 시간에 따라 다름 (opponent
+가 변하니까). advantage 네트워크는:
+
+- 새 데이터와 옛 데이터가 다른 distribution.
+- 학습이 진동.
+- advantage_loss가 안정적으로 줄지 않고 plateau 또는 증가.
+
+### 5. Strategy Mode Collapse
+
+같은 네트워크끼리 self-play하면 mixed strategy가 deterministic-like한
+하나의 mode로 수렴하기 쉽다 (game-theoretic 의미에서 Nash가 mixed인 경우
+에도). 이는 imperfect information game (Lost Cities 포함) 에서 본질적으로
+suboptimal — exploitable.
+
+## 왜 self_play_league는 안정적인가
+
+| | opponent_network | self_play_league |
+|---|---|---|
+| Opponent policy | **moving** (현재 네트워크) | **fixed** (snapshot pool) |
+| Diversity | 없음 (echo) | 다수 snapshot에서 sampling |
+| Regret estimation | non-stationary | quasi-stationary |
+| 이론적 수렴 | 보장 없음 | average strategy → Nash |
+
+`self_play.max_snapshots > 0` 으로 과거 정책의 스냅샷을 pool에 저장하고
+weighted sampling으로 opponent를 선택하면, 위 문제 4개 (1, 2, 3, 4) 모두
+완화되거나 해결된다.
+
+## 직접 비교: 동일 조건의 self_play_league run
+
+`runs/deep_cfr/2026-05-07_512x3_argmax_tiebreak_2x_updates_10000iter`는
+512x3 opponent_network 실험과 **거의 동일한 hyperparameter**에 단지
+`opponent_policy`만 `self_play_league`로 바꾼 run이다 (10000-iter cap이라
+iter 579에서 멈춰있음).
+
+| 항목 | opponent_network 512x3 | self_play_league 512x3 |
+|------|------------------------|-------------------------|
+| `opponent_policy` | **network** | **self_play_league** |
+| `self_play.max_snapshots` | 0 | **20** |
+| `self_play.current_weight` | 1.0 | 0.5 |
+| `self_play.recent_weight` | 0.0 | 0.3 |
+| `self_play.older_weight` | 0.0 | 0.2 |
+| network / traversal / optimization 나머지 | 동일 | 동일 |
+| `regret_matching.all_negative_fallback` | argmax_tiebreak | argmax_tiebreak |
+| `traversals_per_iteration` | 2 | 2 |
+
+### iteration별 비교
+
+| iter | NETWORK Random WR | NETWORK Safe WR | LEAGUE Random WR | LEAGUE Safe WR |
+|------|-------------------|------------------|-------------------|-----------------|
+| 10 | 78% Δ+28.6 | 8% Δ-40.8 | **86%** Δ+32.0 | 5% Δ-52.7 |
+| 15 | **85%** Δ+32.2 | 5% Δ-47.2 | 70% Δ+14.4 | 7% Δ-53.6 |
+| 30 | 36% Δ-14.3 | 3% Δ-106 | 53% Δ+3.9 | **10%** Δ-75.5 |
+| 100 | 41% Δ-12.9 | 0% Δ-111.6 | **68%** Δ+19.9 | 1% Δ-88.5 |
+| 200 | 33% Δ-17.1 | 1% Δ-114.1 | **63%** Δ+18.9 | 0% Δ-90.7 |
+| 300 | 38% Δ-16.2 | 0% Δ-106.4 | **78%** Δ+35.6 | 1% Δ-74.1 |
+| 350 | 34% Δ-21.0 | 8% Δ-91.9 | **72%** Δ+31.7 | 4% Δ-68.6 |
+
+### Best 비교
+
+| 메트릭 | NETWORK | LEAGUE |
+|--------|---------|--------|
+| Best Random WR | 85% (iter 15) | **86%** (iter 10) |
+| iter 350 시점 Random WR | 34% | **72%** |
+| Best Safe Heuristic WR | 8% (iter 10) | **11%** (iter 555) |
+| 발산 여부 | 발산 (iter 30~) | 발산 없음, plateau 유지 |
+
+### 해석
+
+- **두 run 모두 iter 10–15 근처에서 비슷한 peak (~85%)** 도달.
+  → opponent_policy는 학습 초기 신호에는 영향 없음.
+- 그 이후가 갈림길:
+  - **network**: iter 30부터 발산, iter 100+에서 Random WR ~30–40%로 collapse.
+  - **league**: 같은 시점에 plateau 유지, iter 200–350 사이에 오히려
+    Random WR이 60–78%로 천천히 상승. Safe Heuristic도 매우 느리지만
+    학습 (iter 555에 11%로 best).
+- **iter 350 시점의 Random WR 격차 = 38%p (72% vs 34%)**.
+  같은 hyperparameter, 같은 seed, 같은 traversal/optimization 설정에서
+  opponent policy 하나의 차이가 이런 정도의 격차를 만든다.
+
+이 결과는 본 문서의 "moving target → 발산" 가설을 거의 완벽히 실증한다.
+network는 echo chamber로 무너지고, league는 다양한 fixed snapshot 덕에
+안정적으로 학습을 이어간다.
+
+## 왜 큰 네트워크가 plateau를 늘렸나
+
+가설: 큰 capacity는 더 다양한 strategy mode를 표현할 수 있음. echo
+chamber가 단일 mode로 collapse되는 데 더 오래 걸림. 하지만 일단
+collapse가 시작되면 큰 네트워크도 동일하게 발산. **근본 원인 (moving
+target) 은 capacity로 해결 불가.**
+
+부수 효과: 큰 네트워크 + plateau 동안 **Safe Heuristic 상대 13% WR**은
+다른 어떤 self-play 실험에서도 보지 못한 수치. 이 단계의 checkpoint는
+별도로 보존할 가치가 있을 수 있음 (단, 1024x4 실험은 발산 후 last 만 저장
+되어 iter 85 checkpoint가 archive되지 않았다면 복구 불가).
+
+## 결론 및 권고
+
+1. `traversal.opponent_policy: network`는 **단독으로 사용하지 말 것.**
+   학습 초기에 잘 되는 것처럼 보이다가 발산하므로 짧은 실험으로 위험성을
+   놓치기 쉽다.
+
+2. **기본값은 `self_play_league`** 유지. snapshot pool로 fixed/diverse
+   opponent를 제공해야 안정적.
+
+3. 그래도 `network`를 시도하고 싶다면:
+   - **early stopping** 필수 (eval WR 기준 best checkpoint 보존).
+   - `save_iteration_interval`을 짧게 (예: 5 iter) 설정해서 plateau 시점을
+     archive.
+   - peak 이후 곧바로 중단.
+
+4. 연구 가치 있는 후속 실험:
+   - `opponent_policy: average_strategy` (학습 중인 average 정책 사용).
+     CFR 이론과 더 잘 부합할 가능성.
+   - `opponent_policy: hybrid` (probabilistic mix of network + snapshot).
+     diversity와 simplicity 절충.
+   - 1024x4의 iter 85 plateau 패턴을 self_play_league에서 재현 가능한지.
+
+## 사용된 Config 파일
+
+- `configs/deep_cfr/deep_cfr_opponent_network_512x3_1000iter.yaml`
+- `configs/deep_cfr/deep_cfr_opponent_network_1024x4_1000iter.yaml`
+
+## Run 디렉토리
+
+- `runs/deep_cfr/deep_cfr_opponent_network_512x3_1000iter/` (363 iter에서
+  중단)
+- `runs/deep_cfr/deep_cfr_opponent_network_1024x4_1000iter/` (231 iter에서
+  중단)
+
+각 디렉토리의 `metrics.jsonl`이 본 분석의 raw source.
@@ -0,0 +1,56 @@
+# Deep CFR Profile 2026-05-07
+
+Run directory:
+
+`/mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_024616_deep_cfr_profile_10iter`
+
+Command:
+
+```bash
+uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli train \
+  --config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
+  --checkpoint-dir /mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_024616_deep_cfr_profile_10iter \
+  --max-iterations 10 \
+  --save-latest-only
+```
+
+## Summary
+
+The run completed 10 iterations. Loss values stayed finite.
+
+Non-evaluation iterations were iterations 1-4 and 6-9:
+
+| Metric | Average |
+| --- | ---: |
+| `iteration_seconds` | 9.132692 |
+| `traversal_seconds` | 3.143286 |
+| `memory_add_seconds` | 0.171213 |
+| `advantage_train_seconds` | 5.061859 |
+| `strategy_train_seconds` | 0.911080 |
+| `evaluation_seconds` | 0.000004 |
+| `checkpoint_seconds` | 0.015469 |
+| `batch_tensor_seconds` | 1.613572 |
+
+Evaluation iterations were iterations 5 and 10:
+
+| Metric | Average |
+| --- | ---: |
+| `iteration_seconds` | 22.676999 |
+| `traversal_seconds` | 3.233497 |
+| `memory_add_seconds` | 0.145742 |
+| `advantage_train_seconds` | 7.502700 |
+| `strategy_train_seconds` | 0.915639 |
+| `evaluation_seconds` | 11.004282 |
+| `checkpoint_seconds` | 0.019737 |
+| `batch_tensor_seconds` | 1.704181 |
+
+## Per-Iteration Notes
+
+`advantage_memory_size` grew from 43,019 at iteration 1 to 204,903 at
+iteration 10.
+
+`advantage_player_0_sample_seconds + advantage_player_1_sample_seconds` grew
+from about 0.521s at iteration 1 to about 8.231s at iteration 10.
+
+Traversal stayed near 3 seconds per iteration after iteration 1, except for
+normal run-to-run variance.
@@ -0,0 +1,61 @@
+# Deep CFR Advantage Memory Split Profile 2026-05-07
+
+Run directory:
+
+`/mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_025639_deep_cfr_profile_adv_memory_split_10iter`
+
+Command:
+
+```bash
+uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli train \
+  --config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
+  --checkpoint-dir /mnt/2tbhdd/coolrl-lost-cities-runs/2026-05-07_025639_deep_cfr_profile_adv_memory_split_10iter \
+  --max-iterations 10 \
+  --save-latest-only
+```
+
+## Summary
+
+The run completed 10 iterations. Loss values stayed finite.
+
+Non-evaluation iterations were iterations 1-4 and 6-9:
+
+| Metric | Before | After |
+| --- | ---: | ---: |
+| `iteration_seconds` | 9.132692 | 5.832958 |
+| `traversal_seconds` | 3.143286 | 3.160975 |
+| `memory_add_seconds` | 0.171213 | 0.157649 |
+| `advantage_train_seconds` | 5.061859 | 1.742895 |
+| `strategy_train_seconds` | 0.911080 | 0.912324 |
+| `evaluation_seconds` | 0.000004 | 0.000004 |
+| `checkpoint_seconds` | 0.015469 | 0.015756 |
+| `batch_tensor_seconds` | 1.613572 | 1.553183 |
+| `advantage_player_0_sample_seconds` | 1.652320 | 0.054032 |
+| `advantage_player_1_sample_seconds` | 1.625927 | 0.053600 |
+| `strategy_sample_seconds` | 0.061489 | 0.063131 |
+
+Evaluation iterations were iterations 5 and 10:
+
+| Metric | Before | After |
+| --- | ---: | ---: |
+| `iteration_seconds` | 22.676999 | 16.568693 |
+| `traversal_seconds` | 3.233497 | 3.051501 |
+| `memory_add_seconds` | 0.145742 | 0.146042 |
+| `advantage_train_seconds` | 7.502700 | 1.778785 |
+| `strategy_train_seconds` | 0.915639 | 0.930619 |
+| `evaluation_seconds` | 11.004282 | 10.786219 |
+| `checkpoint_seconds` | 0.019737 | 0.020205 |
+| `batch_tensor_seconds` | 1.704181 | 1.633090 |
+| `advantage_player_0_sample_seconds` | 2.839880 | 0.060441 |
+| `advantage_player_1_sample_seconds` | 2.778888 | 0.061004 |
+| `strategy_sample_seconds` | 0.067247 | 0.068568 |
+
+## Per-Iteration Notes
+
+At iteration 10, `advantage_player_0_sample_seconds +
+advantage_player_1_sample_seconds` changed from about 8.231s to about 0.124s.
+
+At iteration 10, `advantage_train_seconds` changed from about 10.230s to about
+1.782s.
+
+Traversal time stayed close to the previous profile.
@@ -0,0 +1,103 @@
+# Deep CFR Regret Fallback Audit, 2026-05-07
+
+Goal: test whether all-negative regret matching fallback is a plausible source of
+early over-opening in Lost Cities Deep CFR.
+
+## Code Changes
+
+- Added traversal audit metrics for regret matching fallback decisions.
+- Default behavior remains unchanged: `regret_matching.all_negative_fallback: uniform`.
+- Added optional fallback mode: `argmax_tiebreak`.
+- Added CLI override: `--regret-fallback uniform|argmax_tiebreak`.
+
+Key metrics:
+
+- `traversal_regret_matching_decisions`
+- `traversal_regret_fallback_count`
+- `traversal_regret_fallback_rate`
+- `traversal_regret_fallback_avg_depth`
+- `traversal_regret_fallback_depth_bucket_<range>`
+- `traversal_regret_fallback_opened_colors_count_<n>`
+- `traversal_regret_fallback_action_open_new`
+- `traversal_regret_fallback_open_new_selected`
+- `traversal_regret_fallback_open_new_selected_rate`
+- `traversal_regret_fallback_legal_actions_mean`
+- `traversal_regret_fallback_legal_open_new_mean`
+- `traversal_regret_fallback_legal_discard_mean`
+- `traversal_regret_fallback_legal_draw_deck_mean`
+- `traversal_regret_fallback_legal_draw_pile_mean`
+- `traversal_regret_fallback_open_new_available_rate`
+- `traversal_regret_fallback_open_new_selection_over_availability`
+- `traversal_regret_fallback_avg_opened_colors_before_action`
+- `traversal_regret_fallback_argmax_tie_rate`
+- `traversal_regret_fallback_argmax_tie_size_mean`
+- `traversal_regret_fallback_argmax_full_tie_rate`
+- `traversal_regret_fallback_open_new_available_color_<color>`
+- `traversal_regret_fallback_open_new_selected_color_<color>`
+
+Implementation note: fallback policy state is captured immediately after the
+network policy is computed. This avoids child recursion overwriting the
+traversal-level fallback flag before the decision is recorded.
+
+## Runs
+
+Baseline long run, analyzed after it had reached iteration 210:
+
+- `runs/deep_cfr/2026-05-07_legacy_align_full_depth_slot_playability`
+
+Short comparison runs:
+
+- `runs/deep_cfr/2026-05-07_regret_fallback_uniform_20iter`
+- `runs/deep_cfr/2026-05-07_regret_fallback_argmax_tiebreak_20iter`
+
+Both short runs used the same base config and seed:
+
+- `configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml`
+- `seed: 79`
+- `iterations: 20`
+- `save_latest_only`
+
+The 20-iteration runs below were collected before the expanded fallback timing,
+legal-action composition, and tie diagnostics were added. They should be treated
+as the first historical audit snapshot. New paired runs are needed to compare
+the expanded metrics.
+
+Instrumentation smoke run:
+
+- `runs/deep_cfr/2026-05-07_regret_fallback_metrics_smoke_1iter_v2`
+
+This run confirms the expanded metrics are emitted to `metrics.jsonl`.
+
+## Iteration 20 Snapshot
+
+| metric | uniform | argmax_tiebreak |
+|---|---:|---:|
+| traversal_regret_matching_decisions | 39,358 | 49,296 |
+| traversal_regret_fallback_count | 18,362 | 7,588 |
+| traversal_regret_fallback_rate | 0.4665 | 0.1539 |
+| traversal_regret_fallback_open_new_selected | 641 | 164 |
+| traversal_regret_fallback_open_new_selected_rate | 0.0349 | 0.0216 |
+| traversal_regret_fallback_avg_opened_colors_before_action | 4.4325 | 4.6118 |
+| eval_random_avg_opened_colors | 2.48 | 2.16 |
+| eval_random_5_color_open_count | 49 | 35 |
+| eval_safe_heuristic_avg_opened_colors | 2.50 | 2.44 |
+| eval_safe_heuristic_5_color_open_count | 50 | 44 |
+| eval_passive_discard_avg_opened_colors | 2.33 | 2.32 |
+| eval_passive_discard_5_color_open_count | 39 | 41 |
+| eval_random_avg_score_diff0 | 42.46 | 33.58 |
+| eval_safe_heuristic_avg_score_diff0 | -52.65 | -58.25 |
+
+## Read
+
+The audit confirms that uniform fallback fires frequently in the early run.
+At iteration 20, almost half of traversal regret-matching decisions use fallback
+under `uniform`.
+
+`argmax_tiebreak` sharply reduces fallback frequency and absolute fallback
+open-new selections in this 20-iteration comparison. It also lowers 5-color
+counts against random and safe heuristic opponents at iteration 20. The effect is
+not uniform across every opponent in this very short run.
+
+This is diagnostic evidence, not enough to promote `argmax_tiebreak` as the
+default. A longer 50-100 iteration paired run is still needed before deciding
+whether this fixes the plateau without hurting policy quality.
@@ -0,0 +1,170 @@
+# Deep CFR v0 Status vs Legacy coolrl
+
+This document tracks the Lost Cities Deep CFR functionality in this repository
+against the legacy implementation in `../coolrl`.
+
+## Implemented In This Repository
+
+### Traversal
+
+Implemented:
+
+1. Recursive `traverse(state, traverser, iteration, depth)` logic.
+2. Terminal value handling.
+3. Traverser vs opponent node behavior.
+4. Advantage-network-driven policies during traversal.
+5. Regret matching over legal actions.
+6. Sampled action recursion with node-value calculation.
+7. Instantaneous regret collection at traverser nodes.
+8. Strategy-memory collection.
+9. Depth and node-budget cutoffs.
+10. Outcome-sampling epsilon.
+11. Sampled action probability correction.
+12. Optional sampled value clipping.
+13. Unsampled regret modes:
+    - `negative_node_value`
+    - `zero`
+14. Score-diff and rollout-based cutoff values.
+15. Random and safe-heuristic cutoff rollout policies.
+16. Deck-draw chance sampling with state restoration.
+
+Important implementation note:
+
+- The active training path now calls `deep_cfr/traversal.pyx`.
+- The old Python recursive `deep_cfr/traverser.py` path has been removed from
+  mainline code.
+- The rules engine (`game.pyx`), encoding (`encoding.pyx`), regret-matching math
+  (`cfr_math.pyx`), and Deep CFR tree-walking loop now have Cython
+  implementations.
+
+### Training And Memory
+
+Implemented:
+
+1. PyTorch advantage networks.
+2. PyTorch strategy network.
+3. Legal-mask-aware advantage loss.
+4. Masked strategy cross-entropy loss.
+5. Reservoir memory with capacity limits.
+6. Batch sampling.
+7. Legal masks stored with samples.
+8. Single-process traversal.
+9. Multiprocessing traversal worker batches.
+10. Worker result merge in the parent trainer process.
+
+### Encoding
+
+Implemented information-state features:
+
+1. Phase flags.
+2. Current player.
+3. Encoded player.
+4. Deck ratio.
+5. Player hand slot features.
+6. Public expedition summaries for both players.
+7. Public discard summaries.
+8. Public card counts.
+9. Total score and score diff features.
+10. Turn ratio.
+11. Pending-discard one-hot.
+12. Legal action mask.
+
+The encoding is still compact compared with the legacy feature set, but it now
+contains the key public board, discard, score, and legal-action information.
+
+### Runtime Operations
+
+Implemented:
+
+1. Checkpoint save/load.
+2. Latest and per-iteration checkpoint files.
+3. Config stored in checkpoints and `config.json`.
+4. Strategy-net policy adapter.
+5. Evaluation against registered classic bots.
+6. Training CLI.
+7. Evaluation CLI.
+8. Traversal benchmark CLI.
+9. Local run files:
+   - `config.json`
+   - `metrics.jsonl`
+   - `runtime_progress.json`
+   - `train.log`
+10. Traversal benchmark metrics.
+11. Self-play league snapshots.
+12. Self-play league opponent selection from stored snapshots.
+13. Safe-heuristic anchor opponent path.
+14. Weighted current/recent/older/anchor self-play league buckets.
+15. Safe-heuristic imitation pretraining.
+16. Policy-gradient fine-tuning.
+17. Single-vs-multiprocessing benchmark comparison.
+
+## Remaining Differences From Legacy coolrl
+
+The main Deep CFR v0 gaps listed earlier are now implemented. Remaining
+differences are mostly experiment-system maturity and legacy-specific research
+extras.
+
+Still smaller than legacy:
+
+1. Config is YAML-first and nested through Pydantic, but only one smoke preset
+   exists under `configs/deep_cfr/`.
+2. Multiprocessing exists, but it is intentionally simple:
+   - no progress callback per worker batch
+   - no hotspot timing profile
+3. Metrics logging exists, but no plotting/status command exists yet.
+4. Checkpoint artifacts are local only; W&B artifact integration is not added.
+5. Legacy visualization helpers are not ported.
+
+These remaining items are not blockers for running and iterating on Deep CFR v0.
+
+## Performance-Critical Gap
+
+This repository was split out to pursue much higher Lost Cities training
+performance. From that perspective, the main remaining gap is not feature
+parity with legacy `../coolrl`; it is the traversal backend.
+
+Current state:
+
+1. `GameState` mutation, legal-action generation, apply/undo, and cached scoring
+   are implemented in Cython.
+2. Information-state encoding and regret matching have Cython modules.
+3. Full Deep CFR traversal now runs through `traversal.pyx`.
+4. PyTorch policy inference and reservoir memory sample materialization still
+   cross the Python boundary.
+5. Traversal is still recursive inside Cython. The Python recursion-limit guard
+   is no longer the main execution path, but an explicit iterative scheduler is
+   still a future optimization.
+
+Recommended performance roadmap:
+
+1. Continue moving the traversal hot path away from Python object boundaries:
+   - C-level legal action enumeration
+   - C-level push/pop undo
+   - terminal, depth cutoff, and node-budget cutoff
+   - traverser/opponent node handling
+   - outcome sampling
+   - sampled action value correction
+   - instantaneous regret calculation
+   - strategy sample collection
+   - traversal stats collection
+2. Reduce Python boundary costs with batched memory writes.
+3. Add batched network inference for policy calls.
+4. Replace the recursive Cython DFS with an explicit Cython traversal scheduler.
+5. Run multiple traversal contexts concurrently so policy-needed states can be
+   encoded and evaluated in batches.
+
+Python iterative traversal is not the preferred performance path. It would
+remove Python recursion-limit risk, but it would keep most Python object and
+callback overhead in the hot loop. For performance, the next serious step is a
+Cython batched iterative traversal scheduler.
+
+## Suggested Next Steps
+
+1. Add batched memory writes from the Cython traversal engine.
+2. Add benchmark output for recursive Cython traversal vs batched iterative
+   traversal once the scheduler exists.
+3. Add batched policy inference.
+4. Add an explicit Cython iterative traversal scheduler.
+5. Add a status/plot command that reads `metrics.jsonl`.
+6. Add worker progress logging and hotspot timing profile.
+7. Add W&B checkpoint artifacts after checkpoint quality is stable.
@@ -0,0 +1,138 @@
+# Deep CFR v0 Plan
+
+The first Deep CFR target is an end-to-end training pipeline that runs on the
+classic Lost Cities game and exercises the Cython game-state hot path. It does
+not need to be a final research-grade implementation.
+
+## Goals
+
+1. Add a real Deep CFR package under
+   `coolrl_lost_cities.games.classic.deep_cfr`.
+2. Keep traversal in Cython so it can call `GameState` C APIs directly:
+   `_legal_actions_c`, `_unified_legal_actions_c`, `push_action`,
+   `pop_action`, score caches, and deck-sampling helpers.
+3. Train minimal PyTorch advantage and strategy networks from traversal output.
+4. Save and load checkpoints for networks, optimizer state, config, and
+   training counters.
+5. Run a small smoke test that completes at least one training iteration on a
+   tiny workload.
+6. Add a small benchmark for Cython rollout/traversal steps per second.
+
+## Non-goals
+
+- Multi-machine distributed training.
+- Highly optimized replay-memory storage.
+- Exploitability calculation.
+- Perfect feature encoding.
+- Full ISMCTS integration.
+- Large experiment orchestration.
+
+## Proposed Package Shape
+
+```text
+src/coolrl_lost_cities/games/classic/deep_cfr/
+  cfr_math.pyx
+  cfr_math.pxd
+  encoding.pyx
+  encoding.pxd
+  traversal.pyx
+  traversal.pxd
+  config.py
+  memory.py
+  networks.py
+  trainer.py
+  checkpoints.py
+```
+
+`cfr_math.pyx` and `encoding.pyx` already exist as scaffolding. The next major
+piece is `traversal.pyx`.
+
+## Chance Handling
+
+Lost Cities has hidden information from deck order and opponent hand contents.
+For v0, model chance by sampling compatible deck order through `GameState`
+mutation instead of cloning Python snapshots.
+
+Initial implementation:
+
+1. Use a deterministic RNG seed per traversal.
+2. Shuffle the remaining deck region with C-level deck swaps.
+3. Apply and undo actions with `push_action` and `pop_action`.
+4. Do not expose opponent hand or exact unseen deck order in the information
+   state encoding.
+
+This is enough for smoke tests and speed work. More precise public-belief or
+particle sampling can come later.
+
+## Encoding v1
+
+The current encoding is intentionally minimal. Deep CFR v0 should expand it
+without leaking hidden information.
+
+Include:
+
+- Current phase.
+- Current player.
+- Traversing player.
+- Player hand.
+- Both players' public expeditions.
+- Public discard piles.
+- Remaining deck ratio.
+- Cached score or score diff.
+- Legal action mask.
+
+Do not include:
+
+- Opponent hand contents.
+- Exact hidden deck order.
+- Any future card identity that the acting player cannot infer.
+
+The encoding should keep a stable `input_dim` for a given config and expose both
+Python wrappers for tests and C-level buffer writes for traversal.
+
+## Traversal v0
+
+The first traversal does not need every Deep CFR detail. It should prove that the
+Cython control flow is viable.
+
+Steps:
+
+1. Add `traversal.pyx/.pxd`.
+2. Implement a deterministic random rollout helper that uses C-level legal
+   actions and `push_action`/`pop_action`.
+3. Add external-sampling Deep CFR traversal using regret matching.
+4. Emit advantage-memory rows with `(info_state, iteration, action_advantages)`.
+5. Emit strategy-memory rows with `(info_state, iteration, action_policy)`.
+6. Add tests for determinism, push/pop restoration, terminal handling, and
+   action legality.
+
+The first version may return Python objects at module boundaries. Inner loops
+should stay Cython-native.
+
+## Python Training Layer
+
+`trainer.py` should own orchestration:
+
+1. Build or load networks.
+2. Run traversal workers.
+3. Append samples to advantage and strategy memories.
+4. Train advantage networks per player.
+5. Train the average strategy network.
+6. Periodically evaluate using the existing policy/evaluation layer.
+7. Periodically checkpoint.
+
+`memory.py` can start simple with bounded Python/NumPy buffers. Replace it only
+after profiling.
+
+## Validation
+
+Required v0 checks:
+
+1. Cython build succeeds.
+2. Unit tests for math, encoding, chance sampling, and traversal restoration.
+3. A tiny one-iteration trainer smoke test.
+4. A benchmark that reports rollout/traversal steps per second.
+
+The first useful benchmark is not model quality. It is whether traversal is
+actually using the Cython hot path instead of rebuilding Python masks and
+snapshots.
@@ -0,0 +1,11 @@
+# Fast Engine Follow-up Optimizations
+
+The current fast engine exposes Python wrappers for testing and debugging, but
+serious traversal code should use the Cython `fast.pxd` API directly.
+
+Deferred work:
+
+1. Consider direct NumPy or feature-buffer output for RL pipelines instead of
+   building Python lists and converting later.
+2. Consider a single contiguous allocation for state arrays after profiling the
+   simpler separate-allocation layout.
@@ -0,0 +1,53 @@
+# Test Coverage Notes
+
+Current quick coverage command, excluding Cython line coverage:
+
+```bash
+uv run --with coverage coverage run --source=src/coolrl_lost_cities -m pytest tests/games/classic
+uv run --with coverage coverage report -m
+uv run --with coverage coverage html -d htmlcov
+```
+
+Latest snapshot:
+
+- `98 passed`
+- `69%` total coverage
+- The report is effectively Python-only because the existing `.pyx` extensions
+  were not built with Cython tracing.
+
+Deferred ideas:
+
+1. Add a tiny coverage helper script if this command becomes common.
+2. Keep normal coverage Python-only by default, so the existing Cython build
+   artifacts stay untouched.
+3. If `.pyx` line coverage becomes useful, run it in a separate `git worktree`
+   or fresh clone. Avoid `build_ext --inplace --force` in the main working tree,
+   because it can overwrite the normal `.so` files with tracing builds.
+4. Gate Cython tracing behind an environment variable such as
+   `CYTHON_COVERAGE=1`.
+5. For tracing builds, enable both Cython line tracing and coverage's Cython
+   plugin:
+
+```python
+compiler_directives={
+    "linetrace": True,
+    # existing directives...
+}
+define_macros=[("CYTHON_TRACE", "1")]
+```
+
+```ini
+[run]
+plugins = Cython.Coverage
+source = src/coolrl_lost_cities
+```
+
+Rough performance expectation:
+
+- Python-only coverage: usually modest overhead.
+- Cython tracing build without coverage: likely noticeable but manageable.
+- Cython tracing plus coverage: can be several times slower, and tight
+  traversal or encoding loops may be much worse.
+
+Practical default: keep the main tree fast and boring. Measure `.pyx` coverage
+only when there is a specific question about the Cython paths.