Auto-derive run dir from experiment_name + timestamp

Drop checkpoint.directory from config — config defines what an experiment
is, not where its outputs go. The CLI now computes the run directory from
run.experiment_name plus a timestamp, defaulting to runs/tmp/ for
throwaway runs and runs/ when --keep is passed.

- Remove CheckpointConfig.directory and DeepCFRConfig.checkpoint_path
- DeepCFRTrainer takes run_dir: Path explicitly
- CLI: add --keep boolean; --resume requires an explicit path (no shortcut)
- Auto path: runs/[tmp/]<YYYY-MM-DD_HHMMSS>_<experiment_name-kebab>/
- Rename 13 configs to kebab-case; strip directory: lines; kebab their
  experiment_name values
- Rewrite AGENTS.md training/run sections; document
  archive/tmp/<flat> layout, --keep, kebab-case scope
- Update tests for new run_dir flow and dropped --resume shortcut

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 16:32:44 +09:00
co-authored by Claude Opus 4.7
parent a177031963
commit acb664c873
19 changed files with 144 additions and 143 deletions
+61 -33
View File
@@ -8,9 +8,14 @@ project environment and Cython extensions are built/loaded consistently.
- `src/coolrl_lost_cities/games/classic/game.pyx`: Cython Lost Cities engine. - `src/coolrl_lost_cities/games/classic/game.pyx`: Cython Lost Cities engine.
- `src/coolrl_lost_cities/games/classic/deep_cfr/`: Deep CFR training, - `src/coolrl_lost_cities/games/classic/deep_cfr/`: Deep CFR training,
traversal, evaluation, analysis, and CLI code. traversal, evaluation, analysis, and CLI code.
- `configs/deep_cfr/`: Deep CFR YAML configs. - `configs/deep_cfr/`: Deep CFR YAML configs (kebab-case filenames).
- `runs/`: generated training runs. This path is gitignored and may be a - `runs/`: generated training runs. Gitignored, may be a symlink to larger
symlink to larger storage. storage. Layout:
- `runs/archive/`: past runs. **Do not modify or delete.**
- `runs/tmp/`: smoke, tests, throwaway. Free to `rm -rf` anytime.
- `runs/<YYYY-MM-DD_HHMMSS>_<kebab-name>/`: real experiments (flat).
Promote a `runs/<...>` directory to `runs/archive/` with a manual `mv`
once analysis is complete.
- `docs/`: profiling notes, migration notes, and experiment documentation. - `docs/`: profiling notes, migration notes, and experiment documentation.
## Core Commands ## Core Commands
@@ -47,63 +52,85 @@ uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli --help
## Deep CFR Training ## Deep CFR Training
Main full config: The CLI auto-derives the run directory from `run.experiment_name` plus a
timestamp. By default runs land under `runs/tmp/`; pass `--keep` for a real
experiment that should live under `runs/`.
Smoke / throwaway run (lands in `runs/tmp/`):
```bash ```bash
uv run lost-cities-deep-cfr train \ uv run lost-cities-deep-cfr train --config configs/deep_cfr/smoke.yaml
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml # → runs/tmp/<YYYY-MM-DD_HHMMSS>_smoke/
``` ```
Unbounded config: Real experiment (lands in `runs/`):
```bash ```bash
uv run lost-cities-deep-cfr train \ uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded.yaml --config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
--keep
# → runs/<YYYY-MM-DD_HHMMSS>_lost-cities-deep-cfr-selfplay-full-depth-slot-playability/
```
Variant of the same config (override slug):
```bash
uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep-cfr-color-shared-attention-512x3.yaml \
--keep \
--set run.experiment_name=color-attn-v2
# → runs/<YYYY-MM-DD_HHMMSS>_color-attn-v2/
``` ```
Short fixed-iteration run: Short fixed-iteration run:
```bash ```bash
uv run lost-cities-deep-cfr train \ uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \ --config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
--set run.max_iterations=100 \ --set run.max_iterations=100 \
--set run.max_minutes=null \ --set run.max_minutes=null \
--set checkpoint.save_every=0 --set checkpoint.save_every=0
``` ```
Use explicit run directories for experiments. Put Deep CFR runs under Resume (path required, no shortcut):
`runs/deep_cfr/` and prefix generated run names with the date:
```bash ```bash
RUN_DIR="runs/deep_cfr/$(date +%Y-%m-%d_%H%M%S)_deep_cfr_experiment_name"
uv run lost-cities-deep-cfr train \ uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \ --config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
--set checkpoint.directory="$RUN_DIR" \ --resume runs/<YYYY-MM-DD_HHMMSS>_<slug>/latest.pt
--set run.max_iterations=100
``` ```
Date-prefixed examples: When `--resume` is given, the trainer reuses the resumed checkpoint's parent
directory; no new timestamped folder is created.
- `runs/deep_cfr/YYYY-MM-DD_HHMMSS_deep_cfr_100iter`
- `runs/deep_cfr/YYYY-MM-DD_HHMMSS_deep_cfr_unbounded`
Useful train controls: Useful train controls:
- `--resume`: resume from `<checkpoint-dir>/latest.pt`. - `--keep`: real experiment, write under `runs/` (default is `runs/tmp/`).
- `--resume PATH`: resume from a specific checkpoint. - `--resume PATH`: resume from a specific checkpoint. `PATH` is required.
- `--set PATH=VALUE`: override config fields. It is repeatable and parses - `--set PATH=VALUE`: override config fields. Repeatable, parses values as
values as YAML, e.g. `--set traversal.num_workers=4` or YAML (e.g. `--set traversal.num_workers=4`, `--set run.max_minutes=null`).
`--set run.max_minutes=null`.
Common `--set` overrides: Common `--set` overrides:
- `--set run.device=cuda`: set the trainer device. - `--set run.device=cuda`: set the trainer device.
- `--set run.experiment_name=foo-v2`: change the slug used in the run dir
name (kebab-case).
- `--set checkpoint.exact_resume=true`: require checkpoint config compatibility. - `--set checkpoint.exact_resume=true`: require checkpoint config compatibility.
- `--set checkpoint.save_latest=false --set checkpoint.save_every=0`: - `--set checkpoint.save_latest=false --set checkpoint.save_every=0`:
disable checkpoint writes. disable checkpoint writes.
- `--set checkpoint.save_every=0`: keep only `latest.pt` (no archives). - `--set checkpoint.save_every=0`: keep only `latest.pt` (no archives).
- `--set checkpoint.save_every=N`: archive every N iterations. - `--set checkpoint.save_every=N`: archive every N iterations.
## Naming Conventions
- **Directory names, run dirs, config filenames, `experiment_name` values**:
kebab-case (`deep-cfr-color-shared-512x3.yaml`,
`runs/2026-05-08_103045_color-attn-v2/`).
- **YAML keys, Python identifiers, config field names**: snake_case
(unchanged: `hidden_size`, `traversals_per_player`, `experiment_name`).
- The CLI converts `run.experiment_name` to a kebab slug when building the
run directory, so values may contain spaces or mixed case.
## Long Runs ## Long Runs
Run long jobs in a real `tmux` session so the user can attach and stop them. Run long jobs in a real `tmux` session so the user can attach and stop them.
@@ -115,7 +142,8 @@ Start a long unbounded run:
tmux new-session -s coolrl-deepcfr-unbounded \ tmux new-session -s coolrl-deepcfr-unbounded \
-c /home/coolguy/dev/coolrl-lost-cities \ -c /home/coolguy/dev/coolrl-lost-cities \
'uv run lost-cities-deep-cfr train \ 'uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded.yaml' --config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability-unbounded.yaml \
--keep'
``` ```
Attach later: Attach later:
@@ -139,7 +167,7 @@ Ctrl+C
Follow logs from another terminal: Follow logs from another terminal:
```bash ```bash
tail -f runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded/train.log tail -f runs/<YYYY-MM-DD_HHMMSS>_<slug>/train.log
``` ```
The unbounded config intentionally has: The unbounded config intentionally has:
@@ -162,7 +190,7 @@ Evaluate a checkpoint:
```bash ```bash
uv run lost-cities-deep-cfr eval \ uv run lost-cities-deep-cfr eval \
--checkpoint runs/deep_cfr/<run-name>/latest.pt \ --checkpoint runs/<run-dir>/latest.pt \
--opponent random \ --opponent random \
--games 100 \ --games 100 \
--device cpu --device cpu
@@ -172,26 +200,26 @@ Save evaluation game records:
```bash ```bash
uv run lost-cities-deep-cfr eval \ uv run lost-cities-deep-cfr eval \
--checkpoint runs/deep_cfr/<run-name>/latest.pt \ --checkpoint runs/<run-dir>/latest.pt \
--opponent random \ --opponent random \
--games 100 \ --games 100 \
--device cpu \ --device cpu \
--save-games runs/deep_cfr/<run-name>/eval_random_games.json --save-games runs/<run-dir>/eval_random_games.json
``` ```
Generate analysis plots from `metrics.jsonl`: Generate analysis plots from `metrics.jsonl`:
```bash ```bash
uv run lost-cities-deep-cfr analyze \ uv run lost-cities-deep-cfr analyze \
--run runs/deep_cfr/<run-name> --run runs/<run-dir>
``` ```
Write plots to a separate directory: Write plots to a separate directory:
```bash ```bash
uv run lost-cities-deep-cfr analyze \ uv run lost-cities-deep-cfr analyze \
--run runs/deep_cfr/<run-name> \ --run runs/<run-dir> \
--output-dir runs/deep_cfr/<run-name>/analysis --output-dir runs/<run-dir>/analysis
``` ```
The analyzer reads `metrics.jsonl` and writes PNG files grouped by diagnostic The analyzer reads `metrics.jsonl` and writes PNG files grouped by diagnostic
@@ -203,7 +231,7 @@ For smoothing controls, run the analyzer module directly:
```bash ```bash
uv run python -m coolrl_lost_cities.games.classic.deep_cfr.analyze \ uv run python -m coolrl_lost_cities.games.classic.deep_cfr.analyze \
--run runs/deep_cfr/<run-name> \ --run runs/<run-dir> \
--smoothing-window 5 --smoothing-window 5
``` ```
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_color_shared_512x3 experiment_name: lost-cities-deep-cfr-color-shared-512x3
seed: 42 seed: 42
max_iterations: 10000 max_iterations: 10000
max_minutes: null max_minutes: null
@@ -85,7 +85,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_color_shared_512x3
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_color_shared_attention_512x3 experiment_name: lost-cities-deep-cfr-color-shared-attention-512x3
seed: 42 seed: 42
max_iterations: 10000 max_iterations: 10000
max_minutes: null max_minutes: null
@@ -85,7 +85,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_color_shared_attention_512x3
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_color_shared_attention_1000iter experiment_name: lost-cities-deep-cfr-color-shared-attention-1000iter
seed: 42 seed: 42
max_iterations: 1000 max_iterations: 1000
max_minutes: null max_minutes: null
@@ -85,7 +85,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_color_shared_attention_exp_1000iter
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_opponent_average_strategy_512x3_1000iter experiment_name: lost-cities-deep-cfr-opponent-average-strategy-512x3-1000iter
seed: 79 seed: 79
max_iterations: 1000 max_iterations: 1000
max_minutes: null max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_opponent_average_strategy_512x3_1000iter
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_opponent_network_1024x4_1000iter experiment_name: lost-cities-deep-cfr-opponent-network-1024x4-1000iter
seed: 79 seed: 79
max_iterations: 1000 max_iterations: 1000
max_minutes: null max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_opponent_network_1024x4_1000iter
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_opponent_network_512x3_1000iter experiment_name: lost-cities-deep-cfr-opponent-network-512x3-1000iter
seed: 79 seed: 79
max_iterations: 1000 max_iterations: 1000
max_minutes: null max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_opponent_network_512x3_1000iter
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_pure_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter experiment_name: lost-cities-deep-cfr-pure-selfplay-full-depth-slot-playability-512x3-2x-updates-10000iter
seed: 79 seed: 79
max_iterations: 10000 max_iterations: 10000
max_minutes: null max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_pure_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_selfplay_anchor_safe_512x3_2x_updates_10000iter experiment_name: lost-cities-deep-cfr-selfplay-anchor-safe-512x3-2x-updates-10000iter
seed: 79 seed: 79
max_iterations: 10000 max_iterations: 10000
max_minutes: null max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_anchor_safe_512x3_2x_updates_10000iter
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-2x-updates-10000iter
seed: 79 seed: 79
max_iterations: 10000 max_iterations: 10000
max_minutes: null max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_lcfr_10000iter experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-lcfr-10000iter
seed: 79 seed: 79
max_iterations: 10000 max_iterations: 10000
max_minutes: null max_minutes: null
@@ -83,7 +83,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_lcfr_10000iter
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_unbounded experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-unbounded
seed: 79 seed: 79
max_iterations: null max_iterations: null
max_minutes: null max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000 strategy_capacity: 2000000
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_unbounded
save_latest: true save_latest: true
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_unbounded experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-unbounded
seed: 79 seed: 79
max_iterations: null max_iterations: null
max_minutes: null max_minutes: null
@@ -86,6 +86,5 @@ evaluation:
- noisy_safe - noisy_safe
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded
save_every: 100 save_every: 100
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run: run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability
seed: 79 seed: 79
max_iterations: null max_iterations: null
max_minutes: 240 max_minutes: 240
@@ -86,6 +86,5 @@ evaluation:
- noisy_safe - noisy_safe
checkpoint: checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability
save_every: 10 save_every: 10
progress_interval_seconds: 20.0 progress_interval_seconds: 20.0
+1 -1
View File
@@ -1,4 +1,5 @@
run: run:
experiment_name: smoke
max_iterations: 1 max_iterations: 1
seed: 1 seed: 1
device: cpu device: cpu
@@ -25,7 +26,6 @@ memory:
strategy_capacity: 1000 strategy_capacity: 1000
checkpoint: checkpoint:
directory: runs/deep_cfr/smoke
save_every: 0 save_every: 0
save_latest: false save_latest: false
@@ -2,6 +2,8 @@ from __future__ import annotations
import argparse import argparse
import json import json
import re
from datetime import datetime
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
@@ -27,7 +29,19 @@ from coolrl_lost_cities.games.classic.deep_cfr.tracking import RunTracker, Wandb
from coolrl_lost_cities.games.classic.deep_cfr.trainer import DeepCFRTrainer from coolrl_lost_cities.games.classic.deep_cfr.trainer import DeepCFRTrainer
from coolrl_lost_cities.games.classic.game import classic_config from coolrl_lost_cities.games.classic.game import classic_config
_RESUME_LATEST = "__latest__" _SLUG_RE = re.compile(r"[^a-z0-9]+")
def _kebab_slug(value: str) -> str:
slug = _SLUG_RE.sub("-", value.strip().lower()).strip("-")
return slug or "run"
def _resolve_run_dir(config: DeepCFRConfig, *, keep: bool) -> Path:
parent = Path("runs") if keep else Path("runs/tmp")
timestamp = datetime.now().strftime("%Y-%m-%d_%H%M%S")
slug = _kebab_slug(config.run.experiment_name)
return parent / f"{timestamp}_{slug}"
def _load_config(path: str | None) -> DeepCFRConfig: def _load_config(path: str | None) -> DeepCFRConfig:
@@ -67,17 +81,6 @@ def _set_path_override(overrides: dict[str, Any], assignment: str) -> None:
cursor[keys[-1]] = value cursor[keys[-1]] = value
def _resolve_resume_path(config: DeepCFRConfig, resume: str | None) -> str | None:
if resume != _RESUME_LATEST:
return resume
latest_path = config.checkpoint_path / "latest.pt"
if not latest_path.exists():
raise FileNotFoundError(
f"--resume was used without a path, but latest checkpoint does not exist: {latest_path}"
)
return str(latest_path)
def _train_overrides_from_args(args: argparse.Namespace) -> dict[str, Any]: def _train_overrides_from_args(args: argparse.Namespace) -> dict[str, Any]:
overrides: dict[str, Any] = {} overrides: dict[str, Any] = {}
for assignment in getattr(args, "config_overrides", None) or (): for assignment in getattr(args, "config_overrides", None) or ():
@@ -89,7 +92,10 @@ def train_command(args: argparse.Namespace) -> None:
config = _load_config(args.config) config = _load_config(args.config)
overrides = _train_overrides_from_args(args) overrides = _train_overrides_from_args(args)
config = _with_overrides(config, overrides) config = _with_overrides(config, overrides)
resume_path = _resolve_resume_path(config, args.resume) if args.resume:
run_dir = Path(args.resume).parent
else:
run_dir = _resolve_run_dir(config, keep=args.keep)
extra_trackers: list[RunTracker] = [] extra_trackers: list[RunTracker] = []
if args.wandb: if args.wandb:
extra_trackers.append( extra_trackers.append(
@@ -98,18 +104,19 @@ def train_command(args: argparse.Namespace) -> None:
name=args.wandb_name or config.run.experiment_name, name=args.wandb_name or config.run.experiment_name,
mode=args.wandb_mode, mode=args.wandb_mode,
config=config.to_dict(), config=config.to_dict(),
run_dir=str(config.checkpoint_path), run_dir=str(run_dir),
tags=list(args.wandb_tag) if args.wandb_tag else None, tags=list(args.wandb_tag) if args.wandb_tag else None,
) )
) )
trainer = DeepCFRTrainer( trainer = DeepCFRTrainer(
config, config,
config.rules.to_lost_cities_config(seed=config.run.seed), config.rules.to_lost_cities_config(seed=config.run.seed),
run_dir=run_dir,
device=config.run.device, device=config.run.device,
extra_trackers=extra_trackers or None, extra_trackers=extra_trackers or None,
) )
if resume_path: if args.resume:
trainer.load_checkpoint(resume_path) trainer.load_checkpoint(args.resume)
trainer.train() trainer.train()
@@ -203,7 +210,15 @@ def main(argv: list[str] | None = None) -> None:
train = subparsers.add_parser("train") train = subparsers.add_parser("train")
train.add_argument("--config") train.add_argument("--config")
train.add_argument("--resume", nargs="?", const=_RESUME_LATEST, default=None) train.add_argument(
"--resume",
help="Path to a checkpoint to resume from (e.g. runs/.../latest.pt).",
)
train.add_argument(
"--keep",
action="store_true",
help="Place run under runs/ instead of runs/tmp/ (use for real experiments).",
)
train.add_argument( train.add_argument(
"--set", "--set",
action="append", action="append",
@@ -217,16 +217,11 @@ class MemoryConfig(StrictModel):
class CheckpointConfig(StrictModel): class CheckpointConfig(StrictModel):
directory: str = "runs/deep_cfr/default"
save_every: int = 1 save_every: int = 1
save_latest: bool = True save_latest: bool = True
progress_interval_seconds: float = 20.0 progress_interval_seconds: float = 20.0
exact_resume: bool = False exact_resume: bool = False
@property
def path(self) -> Path:
return Path(self.directory)
class EvaluationConfig(StrictModel): class EvaluationConfig(StrictModel):
eval_every: int = 50 eval_every: int = 50
@@ -278,10 +273,6 @@ class DeepCFRConfig(StrictModel):
def to_dict(self) -> dict[str, Any]: def to_dict(self) -> dict[str, Any]:
return self.model_dump(mode="json") return self.model_dump(mode="json")
@property
def checkpoint_path(self) -> Path:
return self.checkpoint.path
def config_from_dict(data: Mapping[str, Any]) -> DeepCFRConfig: def config_from_dict(data: Mapping[str, Any]) -> DeepCFRConfig:
return DeepCFRConfig.model_validate(data) return DeepCFRConfig.model_validate(data)
@@ -157,6 +157,7 @@ class DeepCFRTrainer:
config: DeepCFRConfig | None = None, config: DeepCFRConfig | None = None,
game_config: LostCitiesConfig | None = None, game_config: LostCitiesConfig | None = None,
*, *,
run_dir: str | Path | None = None,
device: str = "cpu", device: str = "cpu",
tracker: RunTracker | None = None, tracker: RunTracker | None = None,
extra_trackers: list[RunTracker] | None = None, extra_trackers: list[RunTracker] | None = None,
@@ -200,7 +201,7 @@ class DeepCFRTrainer:
self.strategy_memory = ReservoirMemory(self.config.memory.strategy_capacity) self.strategy_memory = ReservoirMemory(self.config.memory.strategy_capacity)
self.rng = np.random.default_rng(self.config.run.seed + 101) self.rng = np.random.default_rng(self.config.run.seed + 101)
self.iteration = 0 self.iteration = 0
self.run_dir = self.config.checkpoint_path self.run_dir = Path(run_dir) if run_dir is not None else Path("runs/tmp/default")
self.metrics_path = self.run_dir / "metrics.jsonl" self.metrics_path = self.run_dir / "metrics.jsonl"
self.progress_path = self.run_dir / "runtime_progress.json" self.progress_path = self.run_dir / "runtime_progress.json"
self.log_path = self.run_dir / "train.log" self.log_path = self.run_dir / "train.log"
+35 -56
View File
@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
import re import re
from pathlib import Path
import numpy as np import numpy as np
import torch import torch
@@ -14,8 +15,8 @@ from coolrl_lost_cities.games.classic.deep_cfr.benchmark import (
) )
from coolrl_lost_cities.games.classic.deep_cfr.checkpoints import load_checkpoint from coolrl_lost_cities.games.classic.deep_cfr.checkpoints import load_checkpoint
from coolrl_lost_cities.games.classic.deep_cfr.cli import ( from coolrl_lost_cities.games.classic.deep_cfr.cli import (
_RESUME_LATEST, _kebab_slug,
_resolve_resume_path, _resolve_run_dir,
_train_overrides_from_args, _train_overrides_from_args,
_with_overrides, _with_overrides,
) )
@@ -36,13 +37,13 @@ def test_deep_cfr_loads_smoke_yaml_config() -> None:
assert config.run.max_iterations == 1 assert config.run.max_iterations == 1
assert config.network.hidden_size == 16 assert config.network.hidden_size == 16
assert config.traversal.traversals_per_player == 1 assert config.traversal.traversals_per_player == 1
assert config.checkpoint.directory == "runs/deep_cfr/smoke" assert config.run.experiment_name == "smoke"
def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None: def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
config = load_config("configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml") config = load_config("configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml")
assert config.run.experiment_name.endswith("slot_playability") assert config.run.experiment_name.endswith("slot-playability")
assert config.run.seed == 79 assert config.run.seed == 79
assert config.run.max_iterations is None assert config.run.max_iterations is None
assert config.run.max_minutes == 240 assert config.run.max_minutes == 240
@@ -69,9 +70,6 @@ def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
assert config.regret_matching.all_negative_fallback == "uniform" assert config.regret_matching.all_negative_fallback == "uniform"
assert config.training_weighting.mode == "none" assert config.training_weighting.mode == "none"
assert config.checkpoint.save_every == 10 assert config.checkpoint.save_every == 10
assert (
config.checkpoint.directory == "runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability"
)
def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None: def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None:
@@ -91,7 +89,7 @@ def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None
], ],
}, },
)() )()
config = load_config("configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml") config = load_config("configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml")
overridden = _with_overrides(config, _train_overrides_from_args(args)) overridden = _with_overrides(config, _train_overrides_from_args(args))
@@ -216,25 +214,18 @@ def test_deep_cfr_batched_evaluation_matches_batch_size_one() -> None:
assert np.isclose(batched["policy_entropy"], batch_one["policy_entropy"]) assert np.isclose(batched["policy_entropy"], batch_one["policy_entropy"])
def test_deep_cfr_resume_latest_resolution_uses_config_checkpoint_dir(tmp_path) -> None: def test_deep_cfr_resolve_run_dir_uses_keep_flag_and_kebab_slug() -> None:
config = _deep_cfr_config({"checkpoint": {"directory": str(tmp_path)}}) config = _deep_cfr_config({"run": {"experiment_name": "Color Shared Attn v2"}})
latest = tmp_path / "latest.pt"
latest.write_bytes(b"checkpoint")
assert _resolve_resume_path(config, _RESUME_LATEST) == str(latest) tmp_path = _resolve_run_dir(config, keep=False)
assert _resolve_resume_path(config, "custom.pt") == "custom.pt" keep_path = _resolve_run_dir(config, keep=True)
assert _resolve_resume_path(config, None) is None
assert tmp_path.parent == Path("runs/tmp")
def test_deep_cfr_resume_latest_resolution_requires_latest(tmp_path) -> None: assert keep_path.parent == Path("runs")
config = _deep_cfr_config({"checkpoint": {"directory": str(tmp_path)}}) assert tmp_path.name.endswith("_color-shared-attn-v2")
assert keep_path.name.endswith("_color-shared-attn-v2")
try: assert _kebab_slug("Foo BAR_baz!!") == "foo-bar-baz"
_resolve_resume_path(config, _RESUME_LATEST) assert _kebab_slug(" ") == "run"
except FileNotFoundError as exc:
assert "latest checkpoint does not exist" in str(exc)
else: # pragma: no cover
raise AssertionError("expected FileNotFoundError")
def test_deep_cfr_playability_encoding_extends_input_shape() -> None: def test_deep_cfr_playability_encoding_extends_input_shape() -> None:
@@ -321,13 +312,11 @@ def test_deep_cfr_trainer_forwards_metrics_to_extra_trackers(tmp_path) -> None:
"max_nodes_per_traversal": 16, "max_nodes_per_traversal": 16,
}, },
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2}, "optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": { "checkpoint": {"save_every": 0},
"directory": str(tmp_path / "extra-tracker"),
"save_every": 0,
},
} }
), ),
LostCitiesConfig(seed=71), LostCitiesConfig(seed=71),
run_dir=tmp_path / "extra-tracker",
extra_trackers=[_CaptureTracker()], extra_trackers=[_CaptureTracker()],
) )
@@ -643,14 +632,12 @@ def test_deep_cfr_trainer_saves_loads_and_evaluates_checkpoint(tmp_path) -> None
"max_nodes_per_traversal": 32, "max_nodes_per_traversal": 32,
}, },
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2}, "optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": { "checkpoint": {"save_every": 1},
"directory": str(checkpoint_dir),
"save_every": 1,
},
"evaluation": {"eval_every": 1, "games": 2, "opponents": ("random",)}, "evaluation": {"eval_every": 1, "games": 2, "opponents": ("random",)},
} }
), ),
LostCitiesConfig(seed=41), LostCitiesConfig(seed=41),
run_dir=checkpoint_dir,
) )
metrics = trainer.train() metrics = trainer.train()
@@ -660,13 +647,11 @@ def test_deep_cfr_trainer_saves_loads_and_evaluates_checkpoint(tmp_path) -> None
{ {
"run": {"seed": 41}, "run": {"seed": 41},
"network": {"hidden_size": 16}, "network": {"hidden_size": 16},
"checkpoint": { "checkpoint": {"save_every": 0},
"directory": str(checkpoint_dir),
"save_every": 0,
},
} }
), ),
LostCitiesConfig(seed=41), LostCitiesConfig(seed=41),
run_dir=checkpoint_dir,
) )
restored.load_checkpoint(latest) restored.load_checkpoint(latest)
@@ -701,13 +686,11 @@ def test_deep_cfr_trainer_always_saves_latest_checkpoint(tmp_path) -> None:
"run": {"max_iterations": 1, "seed": 42}, "run": {"max_iterations": 1, "seed": 42},
"network": {"hidden_size": 16}, "network": {"hidden_size": 16},
"traversal": {"traversals_per_player": 1, "max_depth": 1}, "traversal": {"traversals_per_player": 1, "max_depth": 1},
"checkpoint": { "checkpoint": {"save_every": 10},
"directory": str(checkpoint_dir),
"save_every": 10,
},
} }
), ),
LostCitiesConfig(seed=42), LostCitiesConfig(seed=42),
run_dir=checkpoint_dir,
) )
trainer.train() trainer.train()
@@ -724,20 +707,22 @@ def test_deep_cfr_exact_resume_is_explicitly_not_implemented(tmp_path) -> None:
"run": {"max_iterations": 1, "seed": 44}, "run": {"max_iterations": 1, "seed": 44},
"network": {"hidden_size": 16}, "network": {"hidden_size": 16},
"traversal": {"traversals_per_player": 1, "max_depth": 1}, "traversal": {"traversals_per_player": 1, "max_depth": 1},
"checkpoint": {"directory": str(checkpoint_dir), "save_every": 1}, "checkpoint": {"save_every": 1},
} }
), ),
LostCitiesConfig(seed=44), LostCitiesConfig(seed=44),
run_dir=checkpoint_dir,
) )
trainer.train() trainer.train()
exact = DeepCFRTrainer( exact = DeepCFRTrainer(
_deep_cfr_config( _deep_cfr_config(
{ {
"network": {"hidden_size": 16}, "network": {"hidden_size": 16},
"checkpoint": {"directory": str(checkpoint_dir), "exact_resume": True}, "checkpoint": {"exact_resume": True},
} }
), ),
LostCitiesConfig(seed=44), LostCitiesConfig(seed=44),
run_dir=checkpoint_dir,
) )
try: try:
@@ -763,13 +748,11 @@ def test_deep_cfr_trainer_multiprocessing_smoke_run(tmp_path) -> None:
"progress_every_traversals": 1, "progress_every_traversals": 1,
}, },
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2}, "optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": { "checkpoint": {"save_every": 0},
"directory": str(tmp_path / "mp"),
"save_every": 0,
},
} }
), ),
LostCitiesConfig(seed=43), LostCitiesConfig(seed=43),
run_dir=tmp_path / "mp",
) )
metrics = trainer.train() metrics = trainer.train()
@@ -827,13 +810,11 @@ def test_deep_cfr_self_play_league_records_snapshots(tmp_path) -> None:
"anchor_probability": 1.0, "anchor_probability": 1.0,
}, },
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2}, "optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": { "checkpoint": {"save_every": 0},
"directory": str(tmp_path / "league"),
"save_every": 0,
},
} }
), ),
LostCitiesConfig(seed=53), LostCitiesConfig(seed=53),
run_dir=tmp_path / "league",
) )
metrics = trainer.train() metrics = trainer.train()
@@ -864,13 +845,11 @@ def test_deep_cfr_weighted_self_play_league_uses_snapshot_bucket(tmp_path) -> No
"recent_window": 1, "recent_window": 1,
}, },
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2}, "optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": { "checkpoint": {"save_every": 0},
"directory": str(tmp_path / "weighted-league"),
"save_every": 0,
},
} }
), ),
LostCitiesConfig(seed=59), LostCitiesConfig(seed=59),
run_dir=tmp_path / "weighted-league",
) )
metrics = trainer.train() metrics = trainer.train()