Auto-derive run dir from experiment_name + timestamp
Drop checkpoint.directory from config — config defines what an experiment is, not where its outputs go. The CLI now computes the run directory from run.experiment_name plus a timestamp, defaulting to runs/tmp/ for throwaway runs and runs/ when --keep is passed. - Remove CheckpointConfig.directory and DeepCFRConfig.checkpoint_path - DeepCFRTrainer takes run_dir: Path explicitly - CLI: add --keep boolean; --resume requires an explicit path (no shortcut) - Auto path: runs/[tmp/]<YYYY-MM-DD_HHMMSS>_<experiment_name-kebab>/ - Rename 13 configs to kebab-case; strip directory: lines; kebab their experiment_name values - Rewrite AGENTS.md training/run sections; document archive/tmp/<flat> layout, --keep, kebab-case scope - Update tests for new run_dir flow and dropped --resume shortcut Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -8,9 +8,14 @@ project environment and Cython extensions are built/loaded consistently.
|
||||
- `src/coolrl_lost_cities/games/classic/game.pyx`: Cython Lost Cities engine.
|
||||
- `src/coolrl_lost_cities/games/classic/deep_cfr/`: Deep CFR training,
|
||||
traversal, evaluation, analysis, and CLI code.
|
||||
- `configs/deep_cfr/`: Deep CFR YAML configs.
|
||||
- `runs/`: generated training runs. This path is gitignored and may be a
|
||||
symlink to larger storage.
|
||||
- `configs/deep_cfr/`: Deep CFR YAML configs (kebab-case filenames).
|
||||
- `runs/`: generated training runs. Gitignored, may be a symlink to larger
|
||||
storage. Layout:
|
||||
- `runs/archive/`: past runs. **Do not modify or delete.**
|
||||
- `runs/tmp/`: smoke, tests, throwaway. Free to `rm -rf` anytime.
|
||||
- `runs/<YYYY-MM-DD_HHMMSS>_<kebab-name>/`: real experiments (flat).
|
||||
Promote a `runs/<...>` directory to `runs/archive/` with a manual `mv`
|
||||
once analysis is complete.
|
||||
- `docs/`: profiling notes, migration notes, and experiment documentation.
|
||||
|
||||
## Core Commands
|
||||
@@ -47,63 +52,85 @@ uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli --help
|
||||
|
||||
## Deep CFR Training
|
||||
|
||||
Main full config:
|
||||
The CLI auto-derives the run directory from `run.experiment_name` plus a
|
||||
timestamp. By default runs land under `runs/tmp/`; pass `--keep` for a real
|
||||
experiment that should live under `runs/`.
|
||||
|
||||
Smoke / throwaway run (lands in `runs/tmp/`):
|
||||
|
||||
```bash
|
||||
uv run lost-cities-deep-cfr train \
|
||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml
|
||||
uv run lost-cities-deep-cfr train --config configs/deep_cfr/smoke.yaml
|
||||
# → runs/tmp/<YYYY-MM-DD_HHMMSS>_smoke/
|
||||
```
|
||||
|
||||
Unbounded config:
|
||||
Real experiment (lands in `runs/`):
|
||||
|
||||
```bash
|
||||
uv run lost-cities-deep-cfr train \
|
||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded.yaml
|
||||
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
|
||||
--keep
|
||||
# → runs/<YYYY-MM-DD_HHMMSS>_lost-cities-deep-cfr-selfplay-full-depth-slot-playability/
|
||||
```
|
||||
|
||||
Variant of the same config (override slug):
|
||||
|
||||
```bash
|
||||
uv run lost-cities-deep-cfr train \
|
||||
--config configs/deep_cfr/deep-cfr-color-shared-attention-512x3.yaml \
|
||||
--keep \
|
||||
--set run.experiment_name=color-attn-v2
|
||||
# → runs/<YYYY-MM-DD_HHMMSS>_color-attn-v2/
|
||||
```
|
||||
|
||||
Short fixed-iteration run:
|
||||
|
||||
```bash
|
||||
uv run lost-cities-deep-cfr train \
|
||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
|
||||
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
|
||||
--set run.max_iterations=100 \
|
||||
--set run.max_minutes=null \
|
||||
--set checkpoint.save_every=0
|
||||
```
|
||||
|
||||
Use explicit run directories for experiments. Put Deep CFR runs under
|
||||
`runs/deep_cfr/` and prefix generated run names with the date:
|
||||
Resume (path required, no shortcut):
|
||||
|
||||
```bash
|
||||
RUN_DIR="runs/deep_cfr/$(date +%Y-%m-%d_%H%M%S)_deep_cfr_experiment_name"
|
||||
uv run lost-cities-deep-cfr train \
|
||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
|
||||
--set checkpoint.directory="$RUN_DIR" \
|
||||
--set run.max_iterations=100
|
||||
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
|
||||
--resume runs/<YYYY-MM-DD_HHMMSS>_<slug>/latest.pt
|
||||
```
|
||||
|
||||
Date-prefixed examples:
|
||||
|
||||
- `runs/deep_cfr/YYYY-MM-DD_HHMMSS_deep_cfr_100iter`
|
||||
- `runs/deep_cfr/YYYY-MM-DD_HHMMSS_deep_cfr_unbounded`
|
||||
When `--resume` is given, the trainer reuses the resumed checkpoint's parent
|
||||
directory; no new timestamped folder is created.
|
||||
|
||||
Useful train controls:
|
||||
|
||||
- `--resume`: resume from `<checkpoint-dir>/latest.pt`.
|
||||
- `--resume PATH`: resume from a specific checkpoint.
|
||||
- `--set PATH=VALUE`: override config fields. It is repeatable and parses
|
||||
values as YAML, e.g. `--set traversal.num_workers=4` or
|
||||
`--set run.max_minutes=null`.
|
||||
- `--keep`: real experiment, write under `runs/` (default is `runs/tmp/`).
|
||||
- `--resume PATH`: resume from a specific checkpoint. `PATH` is required.
|
||||
- `--set PATH=VALUE`: override config fields. Repeatable, parses values as
|
||||
YAML (e.g. `--set traversal.num_workers=4`, `--set run.max_minutes=null`).
|
||||
|
||||
Common `--set` overrides:
|
||||
|
||||
- `--set run.device=cuda`: set the trainer device.
|
||||
- `--set run.experiment_name=foo-v2`: change the slug used in the run dir
|
||||
name (kebab-case).
|
||||
- `--set checkpoint.exact_resume=true`: require checkpoint config compatibility.
|
||||
- `--set checkpoint.save_latest=false --set checkpoint.save_every=0`:
|
||||
disable checkpoint writes.
|
||||
- `--set checkpoint.save_every=0`: keep only `latest.pt` (no archives).
|
||||
- `--set checkpoint.save_every=N`: archive every N iterations.
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
- **Directory names, run dirs, config filenames, `experiment_name` values**:
|
||||
kebab-case (`deep-cfr-color-shared-512x3.yaml`,
|
||||
`runs/2026-05-08_103045_color-attn-v2/`).
|
||||
- **YAML keys, Python identifiers, config field names**: snake_case
|
||||
(unchanged: `hidden_size`, `traversals_per_player`, `experiment_name`).
|
||||
- The CLI converts `run.experiment_name` to a kebab slug when building the
|
||||
run directory, so values may contain spaces or mixed case.
|
||||
|
||||
## Long Runs
|
||||
|
||||
Run long jobs in a real `tmux` session so the user can attach and stop them.
|
||||
@@ -115,7 +142,8 @@ Start a long unbounded run:
|
||||
tmux new-session -s coolrl-deepcfr-unbounded \
|
||||
-c /home/coolguy/dev/coolrl-lost-cities \
|
||||
'uv run lost-cities-deep-cfr train \
|
||||
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded.yaml'
|
||||
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability-unbounded.yaml \
|
||||
--keep'
|
||||
```
|
||||
|
||||
Attach later:
|
||||
@@ -139,7 +167,7 @@ Ctrl+C
|
||||
Follow logs from another terminal:
|
||||
|
||||
```bash
|
||||
tail -f runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded/train.log
|
||||
tail -f runs/<YYYY-MM-DD_HHMMSS>_<slug>/train.log
|
||||
```
|
||||
|
||||
The unbounded config intentionally has:
|
||||
@@ -162,7 +190,7 @@ Evaluate a checkpoint:
|
||||
|
||||
```bash
|
||||
uv run lost-cities-deep-cfr eval \
|
||||
--checkpoint runs/deep_cfr/<run-name>/latest.pt \
|
||||
--checkpoint runs/<run-dir>/latest.pt \
|
||||
--opponent random \
|
||||
--games 100 \
|
||||
--device cpu
|
||||
@@ -172,26 +200,26 @@ Save evaluation game records:
|
||||
|
||||
```bash
|
||||
uv run lost-cities-deep-cfr eval \
|
||||
--checkpoint runs/deep_cfr/<run-name>/latest.pt \
|
||||
--checkpoint runs/<run-dir>/latest.pt \
|
||||
--opponent random \
|
||||
--games 100 \
|
||||
--device cpu \
|
||||
--save-games runs/deep_cfr/<run-name>/eval_random_games.json
|
||||
--save-games runs/<run-dir>/eval_random_games.json
|
||||
```
|
||||
|
||||
Generate analysis plots from `metrics.jsonl`:
|
||||
|
||||
```bash
|
||||
uv run lost-cities-deep-cfr analyze \
|
||||
--run runs/deep_cfr/<run-name>
|
||||
--run runs/<run-dir>
|
||||
```
|
||||
|
||||
Write plots to a separate directory:
|
||||
|
||||
```bash
|
||||
uv run lost-cities-deep-cfr analyze \
|
||||
--run runs/deep_cfr/<run-name> \
|
||||
--output-dir runs/deep_cfr/<run-name>/analysis
|
||||
--run runs/<run-dir> \
|
||||
--output-dir runs/<run-dir>/analysis
|
||||
```
|
||||
|
||||
The analyzer reads `metrics.jsonl` and writes PNG files grouped by diagnostic
|
||||
@@ -203,7 +231,7 @@ For smoothing controls, run the analyzer module directly:
|
||||
|
||||
```bash
|
||||
uv run python -m coolrl_lost_cities.games.classic.deep_cfr.analyze \
|
||||
--run runs/deep_cfr/<run-name> \
|
||||
--run runs/<run-dir> \
|
||||
--smoothing-window 5
|
||||
```
|
||||
|
||||
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_color_shared_512x3
|
||||
experiment_name: lost-cities-deep-cfr-color-shared-512x3
|
||||
seed: 42
|
||||
max_iterations: 10000
|
||||
max_minutes: null
|
||||
@@ -85,7 +85,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_color_shared_512x3
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_color_shared_attention_512x3
|
||||
experiment_name: lost-cities-deep-cfr-color-shared-attention-512x3
|
||||
seed: 42
|
||||
max_iterations: 10000
|
||||
max_minutes: null
|
||||
@@ -85,7 +85,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_color_shared_attention_512x3
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_color_shared_attention_1000iter
|
||||
experiment_name: lost-cities-deep-cfr-color-shared-attention-1000iter
|
||||
seed: 42
|
||||
max_iterations: 1000
|
||||
max_minutes: null
|
||||
@@ -85,7 +85,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_color_shared_attention_exp_1000iter
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_opponent_average_strategy_512x3_1000iter
|
||||
experiment_name: lost-cities-deep-cfr-opponent-average-strategy-512x3-1000iter
|
||||
seed: 79
|
||||
max_iterations: 1000
|
||||
max_minutes: null
|
||||
@@ -82,7 +82,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_opponent_average_strategy_512x3_1000iter
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_opponent_network_1024x4_1000iter
|
||||
experiment_name: lost-cities-deep-cfr-opponent-network-1024x4-1000iter
|
||||
seed: 79
|
||||
max_iterations: 1000
|
||||
max_minutes: null
|
||||
@@ -82,7 +82,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_opponent_network_1024x4_1000iter
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_opponent_network_512x3_1000iter
|
||||
experiment_name: lost-cities-deep-cfr-opponent-network-512x3-1000iter
|
||||
seed: 79
|
||||
max_iterations: 1000
|
||||
max_minutes: null
|
||||
@@ -82,7 +82,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_opponent_network_512x3_1000iter
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_pure_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
|
||||
experiment_name: lost-cities-deep-cfr-pure-selfplay-full-depth-slot-playability-512x3-2x-updates-10000iter
|
||||
seed: 79
|
||||
max_iterations: 10000
|
||||
max_minutes: null
|
||||
@@ -82,7 +82,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_pure_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_selfplay_anchor_safe_512x3_2x_updates_10000iter
|
||||
experiment_name: lost-cities-deep-cfr-selfplay-anchor-safe-512x3-2x-updates-10000iter
|
||||
seed: 79
|
||||
max_iterations: 10000
|
||||
max_minutes: null
|
||||
@@ -82,7 +82,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_selfplay_anchor_safe_512x3_2x_updates_10000iter
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
|
||||
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-2x-updates-10000iter
|
||||
seed: 79
|
||||
max_iterations: 10000
|
||||
max_minutes: null
|
||||
@@ -82,7 +82,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_lcfr_10000iter
|
||||
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-lcfr-10000iter
|
||||
seed: 79
|
||||
max_iterations: 10000
|
||||
max_minutes: null
|
||||
@@ -83,7 +83,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_lcfr_10000iter
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_unbounded
|
||||
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-unbounded
|
||||
seed: 79
|
||||
max_iterations: null
|
||||
max_minutes: null
|
||||
@@ -82,7 +82,6 @@ memory:
|
||||
strategy_capacity: 2000000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_unbounded
|
||||
save_latest: true
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_unbounded
|
||||
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-unbounded
|
||||
seed: 79
|
||||
max_iterations: null
|
||||
max_minutes: null
|
||||
@@ -86,6 +86,5 @@ evaluation:
|
||||
- noisy_safe
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded
|
||||
save_every: 100
|
||||
progress_interval_seconds: 20.0
|
||||
+1
-2
@@ -1,5 +1,5 @@
|
||||
run:
|
||||
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability
|
||||
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability
|
||||
seed: 79
|
||||
max_iterations: null
|
||||
max_minutes: 240
|
||||
@@ -86,6 +86,5 @@ evaluation:
|
||||
- noisy_safe
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability
|
||||
save_every: 10
|
||||
progress_interval_seconds: 20.0
|
||||
@@ -1,4 +1,5 @@
|
||||
run:
|
||||
experiment_name: smoke
|
||||
max_iterations: 1
|
||||
seed: 1
|
||||
device: cpu
|
||||
@@ -25,7 +26,6 @@ memory:
|
||||
strategy_capacity: 1000
|
||||
|
||||
checkpoint:
|
||||
directory: runs/deep_cfr/smoke
|
||||
save_every: 0
|
||||
save_latest: false
|
||||
|
||||
|
||||
@@ -2,6 +2,8 @@ from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -27,7 +29,19 @@ from coolrl_lost_cities.games.classic.deep_cfr.tracking import RunTracker, Wandb
|
||||
from coolrl_lost_cities.games.classic.deep_cfr.trainer import DeepCFRTrainer
|
||||
from coolrl_lost_cities.games.classic.game import classic_config
|
||||
|
||||
_RESUME_LATEST = "__latest__"
|
||||
_SLUG_RE = re.compile(r"[^a-z0-9]+")
|
||||
|
||||
|
||||
def _kebab_slug(value: str) -> str:
|
||||
slug = _SLUG_RE.sub("-", value.strip().lower()).strip("-")
|
||||
return slug or "run"
|
||||
|
||||
|
||||
def _resolve_run_dir(config: DeepCFRConfig, *, keep: bool) -> Path:
|
||||
parent = Path("runs") if keep else Path("runs/tmp")
|
||||
timestamp = datetime.now().strftime("%Y-%m-%d_%H%M%S")
|
||||
slug = _kebab_slug(config.run.experiment_name)
|
||||
return parent / f"{timestamp}_{slug}"
|
||||
|
||||
|
||||
def _load_config(path: str | None) -> DeepCFRConfig:
|
||||
@@ -67,17 +81,6 @@ def _set_path_override(overrides: dict[str, Any], assignment: str) -> None:
|
||||
cursor[keys[-1]] = value
|
||||
|
||||
|
||||
def _resolve_resume_path(config: DeepCFRConfig, resume: str | None) -> str | None:
|
||||
if resume != _RESUME_LATEST:
|
||||
return resume
|
||||
latest_path = config.checkpoint_path / "latest.pt"
|
||||
if not latest_path.exists():
|
||||
raise FileNotFoundError(
|
||||
f"--resume was used without a path, but latest checkpoint does not exist: {latest_path}"
|
||||
)
|
||||
return str(latest_path)
|
||||
|
||||
|
||||
def _train_overrides_from_args(args: argparse.Namespace) -> dict[str, Any]:
|
||||
overrides: dict[str, Any] = {}
|
||||
for assignment in getattr(args, "config_overrides", None) or ():
|
||||
@@ -89,7 +92,10 @@ def train_command(args: argparse.Namespace) -> None:
|
||||
config = _load_config(args.config)
|
||||
overrides = _train_overrides_from_args(args)
|
||||
config = _with_overrides(config, overrides)
|
||||
resume_path = _resolve_resume_path(config, args.resume)
|
||||
if args.resume:
|
||||
run_dir = Path(args.resume).parent
|
||||
else:
|
||||
run_dir = _resolve_run_dir(config, keep=args.keep)
|
||||
extra_trackers: list[RunTracker] = []
|
||||
if args.wandb:
|
||||
extra_trackers.append(
|
||||
@@ -98,18 +104,19 @@ def train_command(args: argparse.Namespace) -> None:
|
||||
name=args.wandb_name or config.run.experiment_name,
|
||||
mode=args.wandb_mode,
|
||||
config=config.to_dict(),
|
||||
run_dir=str(config.checkpoint_path),
|
||||
run_dir=str(run_dir),
|
||||
tags=list(args.wandb_tag) if args.wandb_tag else None,
|
||||
)
|
||||
)
|
||||
trainer = DeepCFRTrainer(
|
||||
config,
|
||||
config.rules.to_lost_cities_config(seed=config.run.seed),
|
||||
run_dir=run_dir,
|
||||
device=config.run.device,
|
||||
extra_trackers=extra_trackers or None,
|
||||
)
|
||||
if resume_path:
|
||||
trainer.load_checkpoint(resume_path)
|
||||
if args.resume:
|
||||
trainer.load_checkpoint(args.resume)
|
||||
trainer.train()
|
||||
|
||||
|
||||
@@ -203,7 +210,15 @@ def main(argv: list[str] | None = None) -> None:
|
||||
|
||||
train = subparsers.add_parser("train")
|
||||
train.add_argument("--config")
|
||||
train.add_argument("--resume", nargs="?", const=_RESUME_LATEST, default=None)
|
||||
train.add_argument(
|
||||
"--resume",
|
||||
help="Path to a checkpoint to resume from (e.g. runs/.../latest.pt).",
|
||||
)
|
||||
train.add_argument(
|
||||
"--keep",
|
||||
action="store_true",
|
||||
help="Place run under runs/ instead of runs/tmp/ (use for real experiments).",
|
||||
)
|
||||
train.add_argument(
|
||||
"--set",
|
||||
action="append",
|
||||
|
||||
@@ -217,16 +217,11 @@ class MemoryConfig(StrictModel):
|
||||
|
||||
|
||||
class CheckpointConfig(StrictModel):
|
||||
directory: str = "runs/deep_cfr/default"
|
||||
save_every: int = 1
|
||||
save_latest: bool = True
|
||||
progress_interval_seconds: float = 20.0
|
||||
exact_resume: bool = False
|
||||
|
||||
@property
|
||||
def path(self) -> Path:
|
||||
return Path(self.directory)
|
||||
|
||||
|
||||
class EvaluationConfig(StrictModel):
|
||||
eval_every: int = 50
|
||||
@@ -278,10 +273,6 @@ class DeepCFRConfig(StrictModel):
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
return self.model_dump(mode="json")
|
||||
|
||||
@property
|
||||
def checkpoint_path(self) -> Path:
|
||||
return self.checkpoint.path
|
||||
|
||||
|
||||
def config_from_dict(data: Mapping[str, Any]) -> DeepCFRConfig:
|
||||
return DeepCFRConfig.model_validate(data)
|
||||
|
||||
@@ -157,6 +157,7 @@ class DeepCFRTrainer:
|
||||
config: DeepCFRConfig | None = None,
|
||||
game_config: LostCitiesConfig | None = None,
|
||||
*,
|
||||
run_dir: str | Path | None = None,
|
||||
device: str = "cpu",
|
||||
tracker: RunTracker | None = None,
|
||||
extra_trackers: list[RunTracker] | None = None,
|
||||
@@ -200,7 +201,7 @@ class DeepCFRTrainer:
|
||||
self.strategy_memory = ReservoirMemory(self.config.memory.strategy_capacity)
|
||||
self.rng = np.random.default_rng(self.config.run.seed + 101)
|
||||
self.iteration = 0
|
||||
self.run_dir = self.config.checkpoint_path
|
||||
self.run_dir = Path(run_dir) if run_dir is not None else Path("runs/tmp/default")
|
||||
self.metrics_path = self.run_dir / "metrics.jsonl"
|
||||
self.progress_path = self.run_dir / "runtime_progress.json"
|
||||
self.log_path = self.run_dir / "train.log"
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
@@ -14,8 +15,8 @@ from coolrl_lost_cities.games.classic.deep_cfr.benchmark import (
|
||||
)
|
||||
from coolrl_lost_cities.games.classic.deep_cfr.checkpoints import load_checkpoint
|
||||
from coolrl_lost_cities.games.classic.deep_cfr.cli import (
|
||||
_RESUME_LATEST,
|
||||
_resolve_resume_path,
|
||||
_kebab_slug,
|
||||
_resolve_run_dir,
|
||||
_train_overrides_from_args,
|
||||
_with_overrides,
|
||||
)
|
||||
@@ -36,13 +37,13 @@ def test_deep_cfr_loads_smoke_yaml_config() -> None:
|
||||
assert config.run.max_iterations == 1
|
||||
assert config.network.hidden_size == 16
|
||||
assert config.traversal.traversals_per_player == 1
|
||||
assert config.checkpoint.directory == "runs/deep_cfr/smoke"
|
||||
assert config.run.experiment_name == "smoke"
|
||||
|
||||
|
||||
def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
|
||||
config = load_config("configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml")
|
||||
config = load_config("configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml")
|
||||
|
||||
assert config.run.experiment_name.endswith("slot_playability")
|
||||
assert config.run.experiment_name.endswith("slot-playability")
|
||||
assert config.run.seed == 79
|
||||
assert config.run.max_iterations is None
|
||||
assert config.run.max_minutes == 240
|
||||
@@ -69,9 +70,6 @@ def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
|
||||
assert config.regret_matching.all_negative_fallback == "uniform"
|
||||
assert config.training_weighting.mode == "none"
|
||||
assert config.checkpoint.save_every == 10
|
||||
assert (
|
||||
config.checkpoint.directory == "runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability"
|
||||
)
|
||||
|
||||
|
||||
def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None:
|
||||
@@ -91,7 +89,7 @@ def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None
|
||||
],
|
||||
},
|
||||
)()
|
||||
config = load_config("configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml")
|
||||
config = load_config("configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml")
|
||||
|
||||
overridden = _with_overrides(config, _train_overrides_from_args(args))
|
||||
|
||||
@@ -216,25 +214,18 @@ def test_deep_cfr_batched_evaluation_matches_batch_size_one() -> None:
|
||||
assert np.isclose(batched["policy_entropy"], batch_one["policy_entropy"])
|
||||
|
||||
|
||||
def test_deep_cfr_resume_latest_resolution_uses_config_checkpoint_dir(tmp_path) -> None:
|
||||
config = _deep_cfr_config({"checkpoint": {"directory": str(tmp_path)}})
|
||||
latest = tmp_path / "latest.pt"
|
||||
latest.write_bytes(b"checkpoint")
|
||||
def test_deep_cfr_resolve_run_dir_uses_keep_flag_and_kebab_slug() -> None:
|
||||
config = _deep_cfr_config({"run": {"experiment_name": "Color Shared Attn v2"}})
|
||||
|
||||
assert _resolve_resume_path(config, _RESUME_LATEST) == str(latest)
|
||||
assert _resolve_resume_path(config, "custom.pt") == "custom.pt"
|
||||
assert _resolve_resume_path(config, None) is None
|
||||
tmp_path = _resolve_run_dir(config, keep=False)
|
||||
keep_path = _resolve_run_dir(config, keep=True)
|
||||
|
||||
|
||||
def test_deep_cfr_resume_latest_resolution_requires_latest(tmp_path) -> None:
|
||||
config = _deep_cfr_config({"checkpoint": {"directory": str(tmp_path)}})
|
||||
|
||||
try:
|
||||
_resolve_resume_path(config, _RESUME_LATEST)
|
||||
except FileNotFoundError as exc:
|
||||
assert "latest checkpoint does not exist" in str(exc)
|
||||
else: # pragma: no cover
|
||||
raise AssertionError("expected FileNotFoundError")
|
||||
assert tmp_path.parent == Path("runs/tmp")
|
||||
assert keep_path.parent == Path("runs")
|
||||
assert tmp_path.name.endswith("_color-shared-attn-v2")
|
||||
assert keep_path.name.endswith("_color-shared-attn-v2")
|
||||
assert _kebab_slug("Foo BAR_baz!!") == "foo-bar-baz"
|
||||
assert _kebab_slug(" ") == "run"
|
||||
|
||||
|
||||
def test_deep_cfr_playability_encoding_extends_input_shape() -> None:
|
||||
@@ -321,13 +312,11 @@ def test_deep_cfr_trainer_forwards_metrics_to_extra_trackers(tmp_path) -> None:
|
||||
"max_nodes_per_traversal": 16,
|
||||
},
|
||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||
"checkpoint": {
|
||||
"directory": str(tmp_path / "extra-tracker"),
|
||||
"save_every": 0,
|
||||
},
|
||||
"checkpoint": {"save_every": 0},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=71),
|
||||
run_dir=tmp_path / "extra-tracker",
|
||||
extra_trackers=[_CaptureTracker()],
|
||||
)
|
||||
|
||||
@@ -643,14 +632,12 @@ def test_deep_cfr_trainer_saves_loads_and_evaluates_checkpoint(tmp_path) -> None
|
||||
"max_nodes_per_traversal": 32,
|
||||
},
|
||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||
"checkpoint": {
|
||||
"directory": str(checkpoint_dir),
|
||||
"save_every": 1,
|
||||
},
|
||||
"checkpoint": {"save_every": 1},
|
||||
"evaluation": {"eval_every": 1, "games": 2, "opponents": ("random",)},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=41),
|
||||
run_dir=checkpoint_dir,
|
||||
)
|
||||
|
||||
metrics = trainer.train()
|
||||
@@ -660,13 +647,11 @@ def test_deep_cfr_trainer_saves_loads_and_evaluates_checkpoint(tmp_path) -> None
|
||||
{
|
||||
"run": {"seed": 41},
|
||||
"network": {"hidden_size": 16},
|
||||
"checkpoint": {
|
||||
"directory": str(checkpoint_dir),
|
||||
"save_every": 0,
|
||||
},
|
||||
"checkpoint": {"save_every": 0},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=41),
|
||||
run_dir=checkpoint_dir,
|
||||
)
|
||||
restored.load_checkpoint(latest)
|
||||
|
||||
@@ -701,13 +686,11 @@ def test_deep_cfr_trainer_always_saves_latest_checkpoint(tmp_path) -> None:
|
||||
"run": {"max_iterations": 1, "seed": 42},
|
||||
"network": {"hidden_size": 16},
|
||||
"traversal": {"traversals_per_player": 1, "max_depth": 1},
|
||||
"checkpoint": {
|
||||
"directory": str(checkpoint_dir),
|
||||
"save_every": 10,
|
||||
},
|
||||
"checkpoint": {"save_every": 10},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=42),
|
||||
run_dir=checkpoint_dir,
|
||||
)
|
||||
|
||||
trainer.train()
|
||||
@@ -724,20 +707,22 @@ def test_deep_cfr_exact_resume_is_explicitly_not_implemented(tmp_path) -> None:
|
||||
"run": {"max_iterations": 1, "seed": 44},
|
||||
"network": {"hidden_size": 16},
|
||||
"traversal": {"traversals_per_player": 1, "max_depth": 1},
|
||||
"checkpoint": {"directory": str(checkpoint_dir), "save_every": 1},
|
||||
"checkpoint": {"save_every": 1},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=44),
|
||||
run_dir=checkpoint_dir,
|
||||
)
|
||||
trainer.train()
|
||||
exact = DeepCFRTrainer(
|
||||
_deep_cfr_config(
|
||||
{
|
||||
"network": {"hidden_size": 16},
|
||||
"checkpoint": {"directory": str(checkpoint_dir), "exact_resume": True},
|
||||
"checkpoint": {"exact_resume": True},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=44),
|
||||
run_dir=checkpoint_dir,
|
||||
)
|
||||
|
||||
try:
|
||||
@@ -763,13 +748,11 @@ def test_deep_cfr_trainer_multiprocessing_smoke_run(tmp_path) -> None:
|
||||
"progress_every_traversals": 1,
|
||||
},
|
||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||
"checkpoint": {
|
||||
"directory": str(tmp_path / "mp"),
|
||||
"save_every": 0,
|
||||
},
|
||||
"checkpoint": {"save_every": 0},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=43),
|
||||
run_dir=tmp_path / "mp",
|
||||
)
|
||||
|
||||
metrics = trainer.train()
|
||||
@@ -827,13 +810,11 @@ def test_deep_cfr_self_play_league_records_snapshots(tmp_path) -> None:
|
||||
"anchor_probability": 1.0,
|
||||
},
|
||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||
"checkpoint": {
|
||||
"directory": str(tmp_path / "league"),
|
||||
"save_every": 0,
|
||||
},
|
||||
"checkpoint": {"save_every": 0},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=53),
|
||||
run_dir=tmp_path / "league",
|
||||
)
|
||||
|
||||
metrics = trainer.train()
|
||||
@@ -864,13 +845,11 @@ def test_deep_cfr_weighted_self_play_league_uses_snapshot_bucket(tmp_path) -> No
|
||||
"recent_window": 1,
|
||||
},
|
||||
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
|
||||
"checkpoint": {
|
||||
"directory": str(tmp_path / "weighted-league"),
|
||||
"save_every": 0,
|
||||
},
|
||||
"checkpoint": {"save_every": 0},
|
||||
}
|
||||
),
|
||||
LostCitiesConfig(seed=59),
|
||||
run_dir=tmp_path / "weighted-league",
|
||||
)
|
||||
|
||||
metrics = trainer.train()
|
||||
|
||||
Reference in New Issue
Block a user