Auto-derive run dir from experiment_name + timestamp
Drop checkpoint.directory from config — config defines what an experiment is, not where its outputs go. The CLI now computes the run directory from run.experiment_name plus a timestamp, defaulting to runs/tmp/ for throwaway runs and runs/ when --keep is passed. - Remove CheckpointConfig.directory and DeepCFRConfig.checkpoint_path - DeepCFRTrainer takes run_dir: Path explicitly - CLI: add --keep boolean; --resume requires an explicit path (no shortcut) - Auto path: runs/[tmp/]<YYYY-MM-DD_HHMMSS>_<experiment_name-kebab>/ - Rename 13 configs to kebab-case; strip directory: lines; kebab their experiment_name values - Rewrite AGENTS.md training/run sections; document archive/tmp/<flat> layout, --keep, kebab-case scope - Update tests for new run_dir flow and dropped --resume shortcut Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -8,9 +8,14 @@ project environment and Cython extensions are built/loaded consistently.
|
|||||||
- `src/coolrl_lost_cities/games/classic/game.pyx`: Cython Lost Cities engine.
|
- `src/coolrl_lost_cities/games/classic/game.pyx`: Cython Lost Cities engine.
|
||||||
- `src/coolrl_lost_cities/games/classic/deep_cfr/`: Deep CFR training,
|
- `src/coolrl_lost_cities/games/classic/deep_cfr/`: Deep CFR training,
|
||||||
traversal, evaluation, analysis, and CLI code.
|
traversal, evaluation, analysis, and CLI code.
|
||||||
- `configs/deep_cfr/`: Deep CFR YAML configs.
|
- `configs/deep_cfr/`: Deep CFR YAML configs (kebab-case filenames).
|
||||||
- `runs/`: generated training runs. This path is gitignored and may be a
|
- `runs/`: generated training runs. Gitignored, may be a symlink to larger
|
||||||
symlink to larger storage.
|
storage. Layout:
|
||||||
|
- `runs/archive/`: past runs. **Do not modify or delete.**
|
||||||
|
- `runs/tmp/`: smoke, tests, throwaway. Free to `rm -rf` anytime.
|
||||||
|
- `runs/<YYYY-MM-DD_HHMMSS>_<kebab-name>/`: real experiments (flat).
|
||||||
|
Promote a `runs/<...>` directory to `runs/archive/` with a manual `mv`
|
||||||
|
once analysis is complete.
|
||||||
- `docs/`: profiling notes, migration notes, and experiment documentation.
|
- `docs/`: profiling notes, migration notes, and experiment documentation.
|
||||||
|
|
||||||
## Core Commands
|
## Core Commands
|
||||||
@@ -47,63 +52,85 @@ uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli --help
|
|||||||
|
|
||||||
## Deep CFR Training
|
## Deep CFR Training
|
||||||
|
|
||||||
Main full config:
|
The CLI auto-derives the run directory from `run.experiment_name` plus a
|
||||||
|
timestamp. By default runs land under `runs/tmp/`; pass `--keep` for a real
|
||||||
|
experiment that should live under `runs/`.
|
||||||
|
|
||||||
|
Smoke / throwaway run (lands in `runs/tmp/`):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run lost-cities-deep-cfr train \
|
uv run lost-cities-deep-cfr train --config configs/deep_cfr/smoke.yaml
|
||||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml
|
# → runs/tmp/<YYYY-MM-DD_HHMMSS>_smoke/
|
||||||
```
|
```
|
||||||
|
|
||||||
Unbounded config:
|
Real experiment (lands in `runs/`):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run lost-cities-deep-cfr train \
|
uv run lost-cities-deep-cfr train \
|
||||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded.yaml
|
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
|
||||||
|
--keep
|
||||||
|
# → runs/<YYYY-MM-DD_HHMMSS>_lost-cities-deep-cfr-selfplay-full-depth-slot-playability/
|
||||||
|
```
|
||||||
|
|
||||||
|
Variant of the same config (override slug):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run lost-cities-deep-cfr train \
|
||||||
|
--config configs/deep_cfr/deep-cfr-color-shared-attention-512x3.yaml \
|
||||||
|
--keep \
|
||||||
|
--set run.experiment_name=color-attn-v2
|
||||||
|
# → runs/<YYYY-MM-DD_HHMMSS>_color-attn-v2/
|
||||||
```
|
```
|
||||||
|
|
||||||
Short fixed-iteration run:
|
Short fixed-iteration run:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run lost-cities-deep-cfr train \
|
uv run lost-cities-deep-cfr train \
|
||||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
|
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
|
||||||
--set run.max_iterations=100 \
|
--set run.max_iterations=100 \
|
||||||
--set run.max_minutes=null \
|
--set run.max_minutes=null \
|
||||||
--set checkpoint.save_every=0
|
--set checkpoint.save_every=0
|
||||||
```
|
```
|
||||||
|
|
||||||
Use explicit run directories for experiments. Put Deep CFR runs under
|
Resume (path required, no shortcut):
|
||||||
`runs/deep_cfr/` and prefix generated run names with the date:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
RUN_DIR="runs/deep_cfr/$(date +%Y-%m-%d_%H%M%S)_deep_cfr_experiment_name"
|
|
||||||
uv run lost-cities-deep-cfr train \
|
uv run lost-cities-deep-cfr train \
|
||||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
|
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
|
||||||
--set checkpoint.directory="$RUN_DIR" \
|
--resume runs/<YYYY-MM-DD_HHMMSS>_<slug>/latest.pt
|
||||||
--set run.max_iterations=100
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Date-prefixed examples:
|
When `--resume` is given, the trainer reuses the resumed checkpoint's parent
|
||||||
|
directory; no new timestamped folder is created.
|
||||||
- `runs/deep_cfr/YYYY-MM-DD_HHMMSS_deep_cfr_100iter`
|
|
||||||
- `runs/deep_cfr/YYYY-MM-DD_HHMMSS_deep_cfr_unbounded`
|
|
||||||
|
|
||||||
Useful train controls:
|
Useful train controls:
|
||||||
|
|
||||||
- `--resume`: resume from `<checkpoint-dir>/latest.pt`.
|
- `--keep`: real experiment, write under `runs/` (default is `runs/tmp/`).
|
||||||
- `--resume PATH`: resume from a specific checkpoint.
|
- `--resume PATH`: resume from a specific checkpoint. `PATH` is required.
|
||||||
- `--set PATH=VALUE`: override config fields. It is repeatable and parses
|
- `--set PATH=VALUE`: override config fields. Repeatable, parses values as
|
||||||
values as YAML, e.g. `--set traversal.num_workers=4` or
|
YAML (e.g. `--set traversal.num_workers=4`, `--set run.max_minutes=null`).
|
||||||
`--set run.max_minutes=null`.
|
|
||||||
|
|
||||||
Common `--set` overrides:
|
Common `--set` overrides:
|
||||||
|
|
||||||
- `--set run.device=cuda`: set the trainer device.
|
- `--set run.device=cuda`: set the trainer device.
|
||||||
|
- `--set run.experiment_name=foo-v2`: change the slug used in the run dir
|
||||||
|
name (kebab-case).
|
||||||
- `--set checkpoint.exact_resume=true`: require checkpoint config compatibility.
|
- `--set checkpoint.exact_resume=true`: require checkpoint config compatibility.
|
||||||
- `--set checkpoint.save_latest=false --set checkpoint.save_every=0`:
|
- `--set checkpoint.save_latest=false --set checkpoint.save_every=0`:
|
||||||
disable checkpoint writes.
|
disable checkpoint writes.
|
||||||
- `--set checkpoint.save_every=0`: keep only `latest.pt` (no archives).
|
- `--set checkpoint.save_every=0`: keep only `latest.pt` (no archives).
|
||||||
- `--set checkpoint.save_every=N`: archive every N iterations.
|
- `--set checkpoint.save_every=N`: archive every N iterations.
|
||||||
|
|
||||||
|
## Naming Conventions
|
||||||
|
|
||||||
|
- **Directory names, run dirs, config filenames, `experiment_name` values**:
|
||||||
|
kebab-case (`deep-cfr-color-shared-512x3.yaml`,
|
||||||
|
`runs/2026-05-08_103045_color-attn-v2/`).
|
||||||
|
- **YAML keys, Python identifiers, config field names**: snake_case
|
||||||
|
(unchanged: `hidden_size`, `traversals_per_player`, `experiment_name`).
|
||||||
|
- The CLI converts `run.experiment_name` to a kebab slug when building the
|
||||||
|
run directory, so values may contain spaces or mixed case.
|
||||||
|
|
||||||
## Long Runs
|
## Long Runs
|
||||||
|
|
||||||
Run long jobs in a real `tmux` session so the user can attach and stop them.
|
Run long jobs in a real `tmux` session so the user can attach and stop them.
|
||||||
@@ -115,7 +142,8 @@ Start a long unbounded run:
|
|||||||
tmux new-session -s coolrl-deepcfr-unbounded \
|
tmux new-session -s coolrl-deepcfr-unbounded \
|
||||||
-c /home/coolguy/dev/coolrl-lost-cities \
|
-c /home/coolguy/dev/coolrl-lost-cities \
|
||||||
'uv run lost-cities-deep-cfr train \
|
'uv run lost-cities-deep-cfr train \
|
||||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded.yaml'
|
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability-unbounded.yaml \
|
||||||
|
--keep'
|
||||||
```
|
```
|
||||||
|
|
||||||
Attach later:
|
Attach later:
|
||||||
@@ -139,7 +167,7 @@ Ctrl+C
|
|||||||
Follow logs from another terminal:
|
Follow logs from another terminal:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
tail -f runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded/train.log
|
tail -f runs/<YYYY-MM-DD_HHMMSS>_<slug>/train.log
|
||||||
```
|
```
|
||||||
|
|
||||||
The unbounded config intentionally has:
|
The unbounded config intentionally has:
|
||||||
@@ -162,7 +190,7 @@ Evaluate a checkpoint:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run lost-cities-deep-cfr eval \
|
uv run lost-cities-deep-cfr eval \
|
||||||
--checkpoint runs/deep_cfr/<run-name>/latest.pt \
|
--checkpoint runs/<run-dir>/latest.pt \
|
||||||
--opponent random \
|
--opponent random \
|
||||||
--games 100 \
|
--games 100 \
|
||||||
--device cpu
|
--device cpu
|
||||||
@@ -172,26 +200,26 @@ Save evaluation game records:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run lost-cities-deep-cfr eval \
|
uv run lost-cities-deep-cfr eval \
|
||||||
--checkpoint runs/deep_cfr/<run-name>/latest.pt \
|
--checkpoint runs/<run-dir>/latest.pt \
|
||||||
--opponent random \
|
--opponent random \
|
||||||
--games 100 \
|
--games 100 \
|
||||||
--device cpu \
|
--device cpu \
|
||||||
--save-games runs/deep_cfr/<run-name>/eval_random_games.json
|
--save-games runs/<run-dir>/eval_random_games.json
|
||||||
```
|
```
|
||||||
|
|
||||||
Generate analysis plots from `metrics.jsonl`:
|
Generate analysis plots from `metrics.jsonl`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run lost-cities-deep-cfr analyze \
|
uv run lost-cities-deep-cfr analyze \
|
||||||
--run runs/deep_cfr/<run-name>
|
--run runs/<run-dir>
|
||||||
```
|
```
|
||||||
|
|
||||||
Write plots to a separate directory:
|
Write plots to a separate directory:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run lost-cities-deep-cfr analyze \
|
uv run lost-cities-deep-cfr analyze \
|
||||||
--run runs/deep_cfr/<run-name> \
|
--run runs/<run-dir> \
|
||||||
--output-dir runs/deep_cfr/<run-name>/analysis
|
--output-dir runs/<run-dir>/analysis
|
||||||
```
|
```
|
||||||
|
|
||||||
The analyzer reads `metrics.jsonl` and writes PNG files grouped by diagnostic
|
The analyzer reads `metrics.jsonl` and writes PNG files grouped by diagnostic
|
||||||
@@ -203,7 +231,7 @@ For smoothing controls, run the analyzer module directly:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run python -m coolrl_lost_cities.games.classic.deep_cfr.analyze \
|
uv run python -m coolrl_lost_cities.games.classic.deep_cfr.analyze \
|
||||||
--run runs/deep_cfr/<run-name> \
|
--run runs/<run-dir> \
|
||||||
--smoothing-window 5
|
--smoothing-window 5
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_color_shared_512x3
|
experiment_name: lost-cities-deep-cfr-color-shared-512x3
|
||||||
seed: 42
|
seed: 42
|
||||||
max_iterations: 10000
|
max_iterations: 10000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -85,7 +85,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_color_shared_512x3
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_color_shared_attention_512x3
|
experiment_name: lost-cities-deep-cfr-color-shared-attention-512x3
|
||||||
seed: 42
|
seed: 42
|
||||||
max_iterations: 10000
|
max_iterations: 10000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -85,7 +85,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_color_shared_attention_512x3
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_color_shared_attention_1000iter
|
experiment_name: lost-cities-deep-cfr-color-shared-attention-1000iter
|
||||||
seed: 42
|
seed: 42
|
||||||
max_iterations: 1000
|
max_iterations: 1000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -85,7 +85,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_color_shared_attention_exp_1000iter
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_opponent_average_strategy_512x3_1000iter
|
experiment_name: lost-cities-deep-cfr-opponent-average-strategy-512x3-1000iter
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: 1000
|
max_iterations: 1000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -82,7 +82,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_opponent_average_strategy_512x3_1000iter
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_opponent_network_1024x4_1000iter
|
experiment_name: lost-cities-deep-cfr-opponent-network-1024x4-1000iter
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: 1000
|
max_iterations: 1000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -82,7 +82,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_opponent_network_1024x4_1000iter
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_opponent_network_512x3_1000iter
|
experiment_name: lost-cities-deep-cfr-opponent-network-512x3-1000iter
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: 1000
|
max_iterations: 1000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -82,7 +82,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_opponent_network_512x3_1000iter
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_pure_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
|
experiment_name: lost-cities-deep-cfr-pure-selfplay-full-depth-slot-playability-512x3-2x-updates-10000iter
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: 10000
|
max_iterations: 10000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -82,7 +82,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_pure_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_selfplay_anchor_safe_512x3_2x_updates_10000iter
|
experiment_name: lost-cities-deep-cfr-selfplay-anchor-safe-512x3-2x-updates-10000iter
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: 10000
|
max_iterations: 10000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -82,7 +82,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_selfplay_anchor_safe_512x3_2x_updates_10000iter
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
|
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-2x-updates-10000iter
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: 10000
|
max_iterations: 10000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -82,7 +82,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_lcfr_10000iter
|
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-lcfr-10000iter
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: 10000
|
max_iterations: 10000
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -83,7 +83,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_lcfr_10000iter
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_unbounded
|
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-unbounded
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: null
|
max_iterations: null
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -82,7 +82,6 @@ memory:
|
|||||||
strategy_capacity: 2000000
|
strategy_capacity: 2000000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_unbounded
|
|
||||||
save_latest: true
|
save_latest: true
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_unbounded
|
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-unbounded
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: null
|
max_iterations: null
|
||||||
max_minutes: null
|
max_minutes: null
|
||||||
@@ -86,6 +86,5 @@ evaluation:
|
|||||||
- noisy_safe
|
- noisy_safe
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded
|
|
||||||
save_every: 100
|
save_every: 100
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
+1
-2
@@ -1,5 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability
|
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability
|
||||||
seed: 79
|
seed: 79
|
||||||
max_iterations: null
|
max_iterations: null
|
||||||
max_minutes: 240
|
max_minutes: 240
|
||||||
@@ -86,6 +86,5 @@ evaluation:
|
|||||||
- noisy_safe
|
- noisy_safe
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability
|
|
||||||
save_every: 10
|
save_every: 10
|
||||||
progress_interval_seconds: 20.0
|
progress_interval_seconds: 20.0
|
||||||
@@ -1,4 +1,5 @@
|
|||||||
run:
|
run:
|
||||||
|
experiment_name: smoke
|
||||||
max_iterations: 1
|
max_iterations: 1
|
||||||
seed: 1
|
seed: 1
|
||||||
device: cpu
|
device: cpu
|
||||||
@@ -25,7 +26,6 @@ memory:
|
|||||||
strategy_capacity: 1000
|
strategy_capacity: 1000
|
||||||
|
|
||||||
checkpoint:
|
checkpoint:
|
||||||
directory: runs/deep_cfr/smoke
|
|
||||||
save_every: 0
|
save_every: 0
|
||||||
save_latest: false
|
save_latest: false
|
||||||
|
|
||||||
|
|||||||
@@ -2,6 +2,8 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
import json
|
import json
|
||||||
|
import re
|
||||||
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -27,7 +29,19 @@ from coolrl_lost_cities.games.classic.deep_cfr.tracking import RunTracker, Wandb
|
|||||||
from coolrl_lost_cities.games.classic.deep_cfr.trainer import DeepCFRTrainer
|
from coolrl_lost_cities.games.classic.deep_cfr.trainer import DeepCFRTrainer
|
||||||
from coolrl_lost_cities.games.classic.game import classic_config
|
from coolrl_lost_cities.games.classic.game import classic_config
|
||||||
|
|
||||||
_RESUME_LATEST = "__latest__"
|
_SLUG_RE = re.compile(r"[^a-z0-9]+")
|
||||||
|
|
||||||
|
|
||||||
|
def _kebab_slug(value: str) -> str:
|
||||||
|
slug = _SLUG_RE.sub("-", value.strip().lower()).strip("-")
|
||||||
|
return slug or "run"
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_run_dir(config: DeepCFRConfig, *, keep: bool) -> Path:
|
||||||
|
parent = Path("runs") if keep else Path("runs/tmp")
|
||||||
|
timestamp = datetime.now().strftime("%Y-%m-%d_%H%M%S")
|
||||||
|
slug = _kebab_slug(config.run.experiment_name)
|
||||||
|
return parent / f"{timestamp}_{slug}"
|
||||||
|
|
||||||
|
|
||||||
def _load_config(path: str | None) -> DeepCFRConfig:
|
def _load_config(path: str | None) -> DeepCFRConfig:
|
||||||
@@ -67,17 +81,6 @@ def _set_path_override(overrides: dict[str, Any], assignment: str) -> None:
|
|||||||
cursor[keys[-1]] = value
|
cursor[keys[-1]] = value
|
||||||
|
|
||||||
|
|
||||||
def _resolve_resume_path(config: DeepCFRConfig, resume: str | None) -> str | None:
|
|
||||||
if resume != _RESUME_LATEST:
|
|
||||||
return resume
|
|
||||||
latest_path = config.checkpoint_path / "latest.pt"
|
|
||||||
if not latest_path.exists():
|
|
||||||
raise FileNotFoundError(
|
|
||||||
f"--resume was used without a path, but latest checkpoint does not exist: {latest_path}"
|
|
||||||
)
|
|
||||||
return str(latest_path)
|
|
||||||
|
|
||||||
|
|
||||||
def _train_overrides_from_args(args: argparse.Namespace) -> dict[str, Any]:
|
def _train_overrides_from_args(args: argparse.Namespace) -> dict[str, Any]:
|
||||||
overrides: dict[str, Any] = {}
|
overrides: dict[str, Any] = {}
|
||||||
for assignment in getattr(args, "config_overrides", None) or ():
|
for assignment in getattr(args, "config_overrides", None) or ():
|
||||||
@@ -89,7 +92,10 @@ def train_command(args: argparse.Namespace) -> None:
|
|||||||
config = _load_config(args.config)
|
config = _load_config(args.config)
|
||||||
overrides = _train_overrides_from_args(args)
|
overrides = _train_overrides_from_args(args)
|
||||||
config = _with_overrides(config, overrides)
|
config = _with_overrides(config, overrides)
|
||||||
resume_path = _resolve_resume_path(config, args.resume)
|
if args.resume:
|
||||||
|
run_dir = Path(args.resume).parent
|
||||||
|
else:
|
||||||
|
run_dir = _resolve_run_dir(config, keep=args.keep)
|
||||||
extra_trackers: list[RunTracker] = []
|
extra_trackers: list[RunTracker] = []
|
||||||
if args.wandb:
|
if args.wandb:
|
||||||
extra_trackers.append(
|
extra_trackers.append(
|
||||||
@@ -98,18 +104,19 @@ def train_command(args: argparse.Namespace) -> None:
|
|||||||
name=args.wandb_name or config.run.experiment_name,
|
name=args.wandb_name or config.run.experiment_name,
|
||||||
mode=args.wandb_mode,
|
mode=args.wandb_mode,
|
||||||
config=config.to_dict(),
|
config=config.to_dict(),
|
||||||
run_dir=str(config.checkpoint_path),
|
run_dir=str(run_dir),
|
||||||
tags=list(args.wandb_tag) if args.wandb_tag else None,
|
tags=list(args.wandb_tag) if args.wandb_tag else None,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
trainer = DeepCFRTrainer(
|
trainer = DeepCFRTrainer(
|
||||||
config,
|
config,
|
||||||
config.rules.to_lost_cities_config(seed=config.run.seed),
|
config.rules.to_lost_cities_config(seed=config.run.seed),
|
||||||
|
run_dir=run_dir,
|
||||||
device=config.run.device,
|
device=config.run.device,
|
||||||
extra_trackers=extra_trackers or None,
|
extra_trackers=extra_trackers or None,
|
||||||
)
|
)
|
||||||
if resume_path:
|
if args.resume:
|
||||||
trainer.load_checkpoint(resume_path)
|
trainer.load_checkpoint(args.resume)
|
||||||
trainer.train()
|
trainer.train()
|
||||||
|
|
||||||
|
|
||||||
@@ -203,7 +210,15 @@ def main(argv: list[str] | None = None) -> None:
|
|||||||
|
|
||||||
train = subparsers.add_parser("train")
|
train = subparsers.add_parser("train")
|
||||||
train.add_argument("--config")
|
train.add_argument("--config")
|
||||||
train.add_argument("--resume", nargs="?", const=_RESUME_LATEST, default=None)
|
train.add_argument(
|
||||||
|
"--resume",
|
||||||
|
help="Path to a checkpoint to resume from (e.g. runs/.../latest.pt).",
|
||||||
|
)
|
||||||
|
train.add_argument(
|
||||||
|
"--keep",
|
||||||
|
action="store_true",
|
||||||
|
help="Place run under runs/ instead of runs/tmp/ (use for real experiments).",
|
||||||
|
)
|
||||||
train.add_argument(
|
train.add_argument(
|
||||||
"--set",
|
"--set",
|
||||||
action="append",
|
action="append",
|
||||||
|
|||||||
@@ -217,16 +217,11 @@ class MemoryConfig(StrictModel):
|
|||||||
|
|
||||||
|
|
||||||
class CheckpointConfig(StrictModel):
|
class CheckpointConfig(StrictModel):
|
||||||
directory: str = "runs/deep_cfr/default"
|
|
||||||
save_every: int = 1
|
save_every: int = 1
|
||||||
save_latest: bool = True
|
save_latest: bool = True
|
||||||
progress_interval_seconds: float = 20.0
|
progress_interval_seconds: float = 20.0
|
||||||
exact_resume: bool = False
|
exact_resume: bool = False
|
||||||
|
|
||||||
@property
|
|
||||||
def path(self) -> Path:
|
|
||||||
return Path(self.directory)
|
|
||||||
|
|
||||||
|
|
||||||
class EvaluationConfig(StrictModel):
|
class EvaluationConfig(StrictModel):
|
||||||
eval_every: int = 50
|
eval_every: int = 50
|
||||||
@@ -278,10 +273,6 @@ class DeepCFRConfig(StrictModel):
|
|||||||
def to_dict(self) -> dict[str, Any]:
|
def to_dict(self) -> dict[str, Any]:
|
||||||
return self.model_dump(mode="json")
|
return self.model_dump(mode="json")
|
||||||
|
|
||||||
@property
|
|
||||||
def checkpoint_path(self) -> Path:
|
|
||||||
return self.checkpoint.path
|
|
||||||
|
|
||||||
|
|
||||||
def config_from_dict(data: Mapping[str, Any]) -> DeepCFRConfig:
|
def config_from_dict(data: Mapping[str, Any]) -> DeepCFRConfig:
|
||||||
return DeepCFRConfig.model_validate(data)
|
return DeepCFRConfig.model_validate(data)
|
||||||
|
|||||||
@@ -157,6 +157,7 @@ class DeepCFRTrainer:
|
|||||||
config: DeepCFRConfig | None = None,
|
config: DeepCFRConfig | None = None,
|
||||||
game_config: LostCitiesConfig | None = None,
|
game_config: LostCitiesConfig | None = None,
|
||||||
*,
|
*,
|
||||||
|
run_dir: str | Path | None = None,
|
||||||
device: str = "cpu",
|
device: str = "cpu",
|
||||||
tracker: RunTracker | None = None,
|
tracker: RunTracker | None = None,
|
||||||
extra_trackers: list[RunTracker] | None = None,
|
extra_trackers: list[RunTracker] | None = None,
|
||||||
@@ -200,7 +201,7 @@ class DeepCFRTrainer:
|
|||||||
self.strategy_memory = ReservoirMemory(self.config.memory.strategy_capacity)
|
self.strategy_memory = ReservoirMemory(self.config.memory.strategy_capacity)
|
||||||
self.rng = np.random.default_rng(self.config.run.seed + 101)
|
self.rng = np.random.default_rng(self.config.run.seed + 101)
|
||||||
self.iteration = 0
|
self.iteration = 0
|
||||||
self.run_dir = self.config.checkpoint_path
|
self.run_dir = Path(run_dir) if run_dir is not None else Path("runs/tmp/default")
|
||||||
self.metrics_path = self.run_dir / "metrics.jsonl"
|
self.metrics_path = self.run_dir / "metrics.jsonl"
|
||||||
self.progress_path = self.run_dir / "runtime_progress.json"
|
self.progress_path = self.run_dir / "runtime_progress.json"
|
||||||
self.log_path = self.run_dir / "train.log"
|
self.log_path = self.run_dir / "train.log"
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import re
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import torch
|
import torch
|
||||||
@@ -14,8 +15,8 @@ from coolrl_lost_cities.games.classic.deep_cfr.benchmark import (
|
|||||||
)
|
)
|
||||||
from coolrl_lost_cities.games.classic.deep_cfr.checkpoints import load_checkpoint
|
from coolrl_lost_cities.games.classic.deep_cfr.checkpoints import load_checkpoint
|
||||||
from coolrl_lost_cities.games.classic.deep_cfr.cli import (
|
from coolrl_lost_cities.games.classic.deep_cfr.cli import (
|
||||||
_RESUME_LATEST,
|
_kebab_slug,
|
||||||
_resolve_resume_path,
|
_resolve_run_dir,
|
||||||
_train_overrides_from_args,
|
_train_overrides_from_args,
|
||||||
_with_overrides,
|
_with_overrides,
|
||||||
)
|
)
|
||||||
@@ -36,13 +37,13 @@ def test_deep_cfr_loads_smoke_yaml_config() -> None:
|
|||||||
assert config.run.max_iterations == 1
|
assert config.run.max_iterations == 1
|
||||||
assert config.network.hidden_size == 16
|
assert config.network.hidden_size == 16
|
||||||
assert config.traversal.traversals_per_player == 1
|
assert config.traversal.traversals_per_player == 1
|
||||||
assert config.checkpoint.directory == "runs/deep_cfr/smoke"
|
assert config.run.experiment_name == "smoke"
|
||||||
|
|
||||||
|
|
||||||
def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
|
def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
|
||||||
config = load_config("configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml")
|
config = load_config("configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml")
|
||||||
|
|
||||||
assert config.run.experiment_name.endswith("slot_playability")
|
assert config.run.experiment_name.endswith("slot-playability")
|
||||||
assert config.run.seed == 79
|
assert config.run.seed == 79
|
||||||
assert config.run.max_iterations is None
|
assert config.run.max_iterations is None
|
||||||
assert config.run.max_minutes == 240
|
assert config.run.max_minutes == 240
|
||||||
@@ -69,9 +70,6 @@ def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
|
|||||||
assert config.regret_matching.all_negative_fallback == "uniform"
|
assert config.regret_matching.all_negative_fallback == "uniform"
|
||||||
assert config.training_weighting.mode == "none"
|
assert config.training_weighting.mode == "none"
|
||||||
assert config.checkpoint.save_every == 10
|
assert config.checkpoint.save_every == 10
|
||||||
assert (
|
|
||||||
config.checkpoint.directory == "runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None:
|
def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None:
|
||||||
@@ -91,7 +89,7 @@ def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None
|
|||||||
],
|
],
|
||||||
},
|
},
|
||||||
)()
|
)()
|
||||||
config = load_config("configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml")
|
config = load_config("configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml")
|
||||||
|
|
||||||
overridden = _with_overrides(config, _train_overrides_from_args(args))
|
overridden = _with_overrides(config, _train_overrides_from_args(args))
|
||||||
|
|
||||||
@@ -216,25 +214,18 @@ def test_deep_cfr_batched_evaluation_matches_batch_size_one() -> None:
|
|||||||
assert np.isclose(batched["policy_entropy"], batch_one["policy_entropy"])
|
assert np.isclose(batched["policy_entropy"], batch_one["policy_entropy"])
|
||||||
|
|
||||||
|
|
||||||
def test_deep_cfr_resume_latest_resolution_uses_config_checkpoint_dir(tmp_path) -> None:
|
def test_deep_cfr_resolve_run_dir_uses_keep_flag_and_kebab_slug() -> None:
|
||||||
config = _deep_cfr_config({"checkpoint": {"directory": str(tmp_path)}})
|
config = _deep_cfr_config({"run": {"experiment_name": "Color Shared Attn v2"}})
|
||||||
latest = tmp_path / "latest.pt"
|
|
||||||
latest.write_bytes(b"checkpoint")
|
|
||||||
|
|
||||||
assert _resolve_resume_path(config, _RESUME_LATEST) == str(latest)
|
tmp_path = _resolve_run_dir(config, keep=False)
|
||||||
assert _resolve_resume_path(config, "custom.pt") == "custom.pt"
|
keep_path = _resolve_run_dir(config, keep=True)
|
||||||
assert _resolve_resume_path(config, None) is None
|
|
||||||
|
|
||||||
|
assert tmp_path.parent == Path("runs/tmp")
|
||||||
def test_deep_cfr_resume_latest_resolution_requires_latest(tmp_path) -> None:
|
assert keep_path.parent == Path("runs")
|
||||||
config = _deep_cfr_config({"checkpoint": {"directory": str(tmp_path)}})
|
assert tmp_path.name.endswith("_color-shared-attn-v2")
|
||||||
|
assert keep_path.name.endswith("_color-shared-attn-v2")
|
||||||
try:
|
assert _kebab_slug("Foo BAR_baz!!") == "foo-bar-baz"
|
||||||
_resolve_resume_path(config, _RESUME_LATEST)
|
assert _kebab_slug(" ") == "run"
|
||||||
except FileNotFoundError as exc:
|
|
||||||
assert "latest checkpoint does not exist" in str(exc)
|
|
||||||
else: # pragma: no cover
|
|
||||||
raise AssertionError("expected FileNotFoundError")
|
|
||||||
|
|
||||||
|
|
||||||
def test_deep_cfr_playability_encoding_extends_input_shape() -> None:
|
def test_deep_cfr_playability_encoding_extends_input_shape() -> None:
|
||||||
@@ -321,13 +312,11 @@ def test_deep_cfr_trainer_forwards_metrics_to_extra_trackers(tmp_path) -> None:
|
|||||||
"max_nodes_per_traversal": 16,
|
"max_nodes_per_traversal": 16,
|
||||||
},
|
},
|
||||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||||
"checkpoint": {
|
"checkpoint": {"save_every": 0},
|
||||||
"directory": str(tmp_path / "extra-tracker"),
|
|
||||||
"save_every": 0,
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=71),
|
LostCitiesConfig(seed=71),
|
||||||
|
run_dir=tmp_path / "extra-tracker",
|
||||||
extra_trackers=[_CaptureTracker()],
|
extra_trackers=[_CaptureTracker()],
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -643,14 +632,12 @@ def test_deep_cfr_trainer_saves_loads_and_evaluates_checkpoint(tmp_path) -> None
|
|||||||
"max_nodes_per_traversal": 32,
|
"max_nodes_per_traversal": 32,
|
||||||
},
|
},
|
||||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||||
"checkpoint": {
|
"checkpoint": {"save_every": 1},
|
||||||
"directory": str(checkpoint_dir),
|
|
||||||
"save_every": 1,
|
|
||||||
},
|
|
||||||
"evaluation": {"eval_every": 1, "games": 2, "opponents": ("random",)},
|
"evaluation": {"eval_every": 1, "games": 2, "opponents": ("random",)},
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=41),
|
LostCitiesConfig(seed=41),
|
||||||
|
run_dir=checkpoint_dir,
|
||||||
)
|
)
|
||||||
|
|
||||||
metrics = trainer.train()
|
metrics = trainer.train()
|
||||||
@@ -660,13 +647,11 @@ def test_deep_cfr_trainer_saves_loads_and_evaluates_checkpoint(tmp_path) -> None
|
|||||||
{
|
{
|
||||||
"run": {"seed": 41},
|
"run": {"seed": 41},
|
||||||
"network": {"hidden_size": 16},
|
"network": {"hidden_size": 16},
|
||||||
"checkpoint": {
|
"checkpoint": {"save_every": 0},
|
||||||
"directory": str(checkpoint_dir),
|
|
||||||
"save_every": 0,
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=41),
|
LostCitiesConfig(seed=41),
|
||||||
|
run_dir=checkpoint_dir,
|
||||||
)
|
)
|
||||||
restored.load_checkpoint(latest)
|
restored.load_checkpoint(latest)
|
||||||
|
|
||||||
@@ -701,13 +686,11 @@ def test_deep_cfr_trainer_always_saves_latest_checkpoint(tmp_path) -> None:
|
|||||||
"run": {"max_iterations": 1, "seed": 42},
|
"run": {"max_iterations": 1, "seed": 42},
|
||||||
"network": {"hidden_size": 16},
|
"network": {"hidden_size": 16},
|
||||||
"traversal": {"traversals_per_player": 1, "max_depth": 1},
|
"traversal": {"traversals_per_player": 1, "max_depth": 1},
|
||||||
"checkpoint": {
|
"checkpoint": {"save_every": 10},
|
||||||
"directory": str(checkpoint_dir),
|
|
||||||
"save_every": 10,
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=42),
|
LostCitiesConfig(seed=42),
|
||||||
|
run_dir=checkpoint_dir,
|
||||||
)
|
)
|
||||||
|
|
||||||
trainer.train()
|
trainer.train()
|
||||||
@@ -724,20 +707,22 @@ def test_deep_cfr_exact_resume_is_explicitly_not_implemented(tmp_path) -> None:
|
|||||||
"run": {"max_iterations": 1, "seed": 44},
|
"run": {"max_iterations": 1, "seed": 44},
|
||||||
"network": {"hidden_size": 16},
|
"network": {"hidden_size": 16},
|
||||||
"traversal": {"traversals_per_player": 1, "max_depth": 1},
|
"traversal": {"traversals_per_player": 1, "max_depth": 1},
|
||||||
"checkpoint": {"directory": str(checkpoint_dir), "save_every": 1},
|
"checkpoint": {"save_every": 1},
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=44),
|
LostCitiesConfig(seed=44),
|
||||||
|
run_dir=checkpoint_dir,
|
||||||
)
|
)
|
||||||
trainer.train()
|
trainer.train()
|
||||||
exact = DeepCFRTrainer(
|
exact = DeepCFRTrainer(
|
||||||
_deep_cfr_config(
|
_deep_cfr_config(
|
||||||
{
|
{
|
||||||
"network": {"hidden_size": 16},
|
"network": {"hidden_size": 16},
|
||||||
"checkpoint": {"directory": str(checkpoint_dir), "exact_resume": True},
|
"checkpoint": {"exact_resume": True},
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=44),
|
LostCitiesConfig(seed=44),
|
||||||
|
run_dir=checkpoint_dir,
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -763,13 +748,11 @@ def test_deep_cfr_trainer_multiprocessing_smoke_run(tmp_path) -> None:
|
|||||||
"progress_every_traversals": 1,
|
"progress_every_traversals": 1,
|
||||||
},
|
},
|
||||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||||
"checkpoint": {
|
"checkpoint": {"save_every": 0},
|
||||||
"directory": str(tmp_path / "mp"),
|
|
||||||
"save_every": 0,
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=43),
|
LostCitiesConfig(seed=43),
|
||||||
|
run_dir=tmp_path / "mp",
|
||||||
)
|
)
|
||||||
|
|
||||||
metrics = trainer.train()
|
metrics = trainer.train()
|
||||||
@@ -827,13 +810,11 @@ def test_deep_cfr_self_play_league_records_snapshots(tmp_path) -> None:
|
|||||||
"anchor_probability": 1.0,
|
"anchor_probability": 1.0,
|
||||||
},
|
},
|
||||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||||
"checkpoint": {
|
"checkpoint": {"save_every": 0},
|
||||||
"directory": str(tmp_path / "league"),
|
|
||||||
"save_every": 0,
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=53),
|
LostCitiesConfig(seed=53),
|
||||||
|
run_dir=tmp_path / "league",
|
||||||
)
|
)
|
||||||
|
|
||||||
metrics = trainer.train()
|
metrics = trainer.train()
|
||||||
@@ -864,13 +845,11 @@ def test_deep_cfr_weighted_self_play_league_uses_snapshot_bucket(tmp_path) -> No
|
|||||||
"recent_window": 1,
|
"recent_window": 1,
|
||||||
},
|
},
|
||||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||||
"checkpoint": {
|
"checkpoint": {"save_every": 0},
|
||||||
"directory": str(tmp_path / "weighted-league"),
|
|
||||||
"save_every": 0,
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
),
|
),
|
||||||
LostCitiesConfig(seed=59),
|
LostCitiesConfig(seed=59),
|
||||||
|
run_dir=tmp_path / "weighted-league",
|
||||||
)
|
)
|
||||||
|
|
||||||
metrics = trainer.train()
|
metrics = trainer.train()
|
||||||
|
|||||||
Reference in New Issue
Block a user