Auto-derive run dir from experiment_name + timestamp

Drop checkpoint.directory from config — config defines what an experiment
is, not where its outputs go. The CLI now computes the run directory from
run.experiment_name plus a timestamp, defaulting to runs/tmp/ for
throwaway runs and runs/ when --keep is passed.

- Remove CheckpointConfig.directory and DeepCFRConfig.checkpoint_path
- DeepCFRTrainer takes run_dir: Path explicitly
- CLI: add --keep boolean; --resume requires an explicit path (no shortcut)
- Auto path: runs/[tmp/]<YYYY-MM-DD_HHMMSS>_<experiment_name-kebab>/
- Rename 13 configs to kebab-case; strip directory: lines; kebab their
  experiment_name values
- Rewrite AGENTS.md training/run sections; document
  archive/tmp/<flat> layout, --keep, kebab-case scope
- Update tests for new run_dir flow and dropped --resume shortcut

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 16:32:44 +09:00
co-authored by Claude Opus 4.7
parent a177031963
commit acb664c873
19 changed files with 144 additions and 143 deletions
+61 -33
View File
@@ -8,9 +8,14 @@ project environment and Cython extensions are built/loaded consistently.
- `src/coolrl_lost_cities/games/classic/game.pyx`: Cython Lost Cities engine.
- `src/coolrl_lost_cities/games/classic/deep_cfr/`: Deep CFR training,
traversal, evaluation, analysis, and CLI code.
- `configs/deep_cfr/`: Deep CFR YAML configs.
- `runs/`: generated training runs. This path is gitignored and may be a
symlink to larger storage.
- `configs/deep_cfr/`: Deep CFR YAML configs (kebab-case filenames).
- `runs/`: generated training runs. Gitignored, may be a symlink to larger
storage. Layout:
- `runs/archive/`: past runs. **Do not modify or delete.**
- `runs/tmp/`: smoke, tests, throwaway. Free to `rm -rf` anytime.
- `runs/<YYYY-MM-DD_HHMMSS>_<kebab-name>/`: real experiments (flat).
Promote a `runs/<...>` directory to `runs/archive/` with a manual `mv`
once analysis is complete.
- `docs/`: profiling notes, migration notes, and experiment documentation.
## Core Commands
@@ -47,63 +52,85 @@ uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli --help
## Deep CFR Training
Main full config:
The CLI auto-derives the run directory from `run.experiment_name` plus a
timestamp. By default runs land under `runs/tmp/`; pass `--keep` for a real
experiment that should live under `runs/`.
Smoke / throwaway run (lands in `runs/tmp/`):
```bash
uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml
uv run lost-cities-deep-cfr train --config configs/deep_cfr/smoke.yaml
# → runs/tmp/<YYYY-MM-DD_HHMMSS>_smoke/
```
Unbounded config:
Real experiment (lands in `runs/`):
```bash
uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded.yaml
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
--keep
# → runs/<YYYY-MM-DD_HHMMSS>_lost-cities-deep-cfr-selfplay-full-depth-slot-playability/
```
Variant of the same config (override slug):
```bash
uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep-cfr-color-shared-attention-512x3.yaml \
--keep \
--set run.experiment_name=color-attn-v2
# → runs/<YYYY-MM-DD_HHMMSS>_color-attn-v2/
```
Short fixed-iteration run:
```bash
uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
--set run.max_iterations=100 \
--set run.max_minutes=null \
--set checkpoint.save_every=0
```
Use explicit run directories for experiments. Put Deep CFR runs under
`runs/deep_cfr/` and prefix generated run names with the date:
Resume (path required, no shortcut):
```bash
RUN_DIR="runs/deep_cfr/$(date +%Y-%m-%d_%H%M%S)_deep_cfr_experiment_name"
uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml \
--set checkpoint.directory="$RUN_DIR" \
--set run.max_iterations=100
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml \
--resume runs/<YYYY-MM-DD_HHMMSS>_<slug>/latest.pt
```
Date-prefixed examples:
- `runs/deep_cfr/YYYY-MM-DD_HHMMSS_deep_cfr_100iter`
- `runs/deep_cfr/YYYY-MM-DD_HHMMSS_deep_cfr_unbounded`
When `--resume` is given, the trainer reuses the resumed checkpoint's parent
directory; no new timestamped folder is created.
Useful train controls:
- `--resume`: resume from `<checkpoint-dir>/latest.pt`.
- `--resume PATH`: resume from a specific checkpoint.
- `--set PATH=VALUE`: override config fields. It is repeatable and parses
values as YAML, e.g. `--set traversal.num_workers=4` or
`--set run.max_minutes=null`.
- `--keep`: real experiment, write under `runs/` (default is `runs/tmp/`).
- `--resume PATH`: resume from a specific checkpoint. `PATH` is required.
- `--set PATH=VALUE`: override config fields. Repeatable, parses values as
YAML (e.g. `--set traversal.num_workers=4`, `--set run.max_minutes=null`).
Common `--set` overrides:
- `--set run.device=cuda`: set the trainer device.
- `--set run.experiment_name=foo-v2`: change the slug used in the run dir
name (kebab-case).
- `--set checkpoint.exact_resume=true`: require checkpoint config compatibility.
- `--set checkpoint.save_latest=false --set checkpoint.save_every=0`:
disable checkpoint writes.
- `--set checkpoint.save_every=0`: keep only `latest.pt` (no archives).
- `--set checkpoint.save_every=N`: archive every N iterations.
## Naming Conventions
- **Directory names, run dirs, config filenames, `experiment_name` values**:
kebab-case (`deep-cfr-color-shared-512x3.yaml`,
`runs/2026-05-08_103045_color-attn-v2/`).
- **YAML keys, Python identifiers, config field names**: snake_case
(unchanged: `hidden_size`, `traversals_per_player`, `experiment_name`).
- The CLI converts `run.experiment_name` to a kebab slug when building the
run directory, so values may contain spaces or mixed case.
## Long Runs
Run long jobs in a real `tmux` session so the user can attach and stop them.
@@ -115,7 +142,8 @@ Start a long unbounded run:
tmux new-session -s coolrl-deepcfr-unbounded \
-c /home/coolguy/dev/coolrl-lost-cities \
'uv run lost-cities-deep-cfr train \
--config configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded.yaml'
--config configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability-unbounded.yaml \
--keep'
```
Attach later:
@@ -139,7 +167,7 @@ Ctrl+C
Follow logs from another terminal:
```bash
tail -f runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded/train.log
tail -f runs/<YYYY-MM-DD_HHMMSS>_<slug>/train.log
```
The unbounded config intentionally has:
@@ -162,7 +190,7 @@ Evaluate a checkpoint:
```bash
uv run lost-cities-deep-cfr eval \
--checkpoint runs/deep_cfr/<run-name>/latest.pt \
--checkpoint runs/<run-dir>/latest.pt \
--opponent random \
--games 100 \
--device cpu
@@ -172,26 +200,26 @@ Save evaluation game records:
```bash
uv run lost-cities-deep-cfr eval \
--checkpoint runs/deep_cfr/<run-name>/latest.pt \
--checkpoint runs/<run-dir>/latest.pt \
--opponent random \
--games 100 \
--device cpu \
--save-games runs/deep_cfr/<run-name>/eval_random_games.json
--save-games runs/<run-dir>/eval_random_games.json
```
Generate analysis plots from `metrics.jsonl`:
```bash
uv run lost-cities-deep-cfr analyze \
--run runs/deep_cfr/<run-name>
--run runs/<run-dir>
```
Write plots to a separate directory:
```bash
uv run lost-cities-deep-cfr analyze \
--run runs/deep_cfr/<run-name> \
--output-dir runs/deep_cfr/<run-name>/analysis
--run runs/<run-dir> \
--output-dir runs/<run-dir>/analysis
```
The analyzer reads `metrics.jsonl` and writes PNG files grouped by diagnostic
@@ -203,7 +231,7 @@ For smoothing controls, run the analyzer module directly:
```bash
uv run python -m coolrl_lost_cities.games.classic.deep_cfr.analyze \
--run runs/deep_cfr/<run-name> \
--run runs/<run-dir> \
--smoothing-window 5
```
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_color_shared_512x3
experiment_name: lost-cities-deep-cfr-color-shared-512x3
seed: 42
max_iterations: 10000
max_minutes: null
@@ -85,7 +85,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_color_shared_512x3
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_color_shared_attention_512x3
experiment_name: lost-cities-deep-cfr-color-shared-attention-512x3
seed: 42
max_iterations: 10000
max_minutes: null
@@ -85,7 +85,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_color_shared_attention_512x3
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_color_shared_attention_1000iter
experiment_name: lost-cities-deep-cfr-color-shared-attention-1000iter
seed: 42
max_iterations: 1000
max_minutes: null
@@ -85,7 +85,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_color_shared_attention_exp_1000iter
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_opponent_average_strategy_512x3_1000iter
experiment_name: lost-cities-deep-cfr-opponent-average-strategy-512x3-1000iter
seed: 79
max_iterations: 1000
max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_opponent_average_strategy_512x3_1000iter
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_opponent_network_1024x4_1000iter
experiment_name: lost-cities-deep-cfr-opponent-network-1024x4-1000iter
seed: 79
max_iterations: 1000
max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_opponent_network_1024x4_1000iter
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_opponent_network_512x3_1000iter
experiment_name: lost-cities-deep-cfr-opponent-network-512x3-1000iter
seed: 79
max_iterations: 1000
max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_opponent_network_512x3_1000iter
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_pure_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
experiment_name: lost-cities-deep-cfr-pure-selfplay-full-depth-slot-playability-512x3-2x-updates-10000iter
seed: 79
max_iterations: 10000
max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_pure_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_selfplay_anchor_safe_512x3_2x_updates_10000iter
experiment_name: lost-cities-deep-cfr-selfplay-anchor-safe-512x3-2x-updates-10000iter
seed: 79
max_iterations: 10000
max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_anchor_safe_512x3_2x_updates_10000iter
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-2x-updates-10000iter
seed: 79
max_iterations: 10000
max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_2x_updates_10000iter
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_lcfr_10000iter
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-lcfr-10000iter
seed: 79
max_iterations: 10000
max_minutes: null
@@ -83,7 +83,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_lcfr_10000iter
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_512x3_unbounded
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-512x3-unbounded
seed: 79
max_iterations: null
max_minutes: null
@@ -82,7 +82,6 @@ memory:
strategy_capacity: 2000000
checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_512x3_unbounded
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability_unbounded
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability-unbounded
seed: 79
max_iterations: null
max_minutes: null
@@ -86,6 +86,5 @@ evaluation:
- noisy_safe
checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability_unbounded
save_every: 100
progress_interval_seconds: 20.0
@@ -1,5 +1,5 @@
run:
experiment_name: lost_cities_deep_cfr_selfplay_full_depth_slot_playability
experiment_name: lost-cities-deep-cfr-selfplay-full-depth-slot-playability
seed: 79
max_iterations: null
max_minutes: 240
@@ -86,6 +86,5 @@ evaluation:
- noisy_safe
checkpoint:
directory: runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability
save_every: 10
progress_interval_seconds: 20.0
+1 -1
View File
@@ -1,4 +1,5 @@
run:
experiment_name: smoke
max_iterations: 1
seed: 1
device: cpu
@@ -25,7 +26,6 @@ memory:
strategy_capacity: 1000
checkpoint:
directory: runs/deep_cfr/smoke
save_every: 0
save_latest: false
@@ -2,6 +2,8 @@ from __future__ import annotations
import argparse
import json
import re
from datetime import datetime
from pathlib import Path
from typing import Any
@@ -27,7 +29,19 @@ from coolrl_lost_cities.games.classic.deep_cfr.tracking import RunTracker, Wandb
from coolrl_lost_cities.games.classic.deep_cfr.trainer import DeepCFRTrainer
from coolrl_lost_cities.games.classic.game import classic_config
_RESUME_LATEST = "__latest__"
_SLUG_RE = re.compile(r"[^a-z0-9]+")
def _kebab_slug(value: str) -> str:
slug = _SLUG_RE.sub("-", value.strip().lower()).strip("-")
return slug or "run"
def _resolve_run_dir(config: DeepCFRConfig, *, keep: bool) -> Path:
parent = Path("runs") if keep else Path("runs/tmp")
timestamp = datetime.now().strftime("%Y-%m-%d_%H%M%S")
slug = _kebab_slug(config.run.experiment_name)
return parent / f"{timestamp}_{slug}"
def _load_config(path: str | None) -> DeepCFRConfig:
@@ -67,17 +81,6 @@ def _set_path_override(overrides: dict[str, Any], assignment: str) -> None:
cursor[keys[-1]] = value
def _resolve_resume_path(config: DeepCFRConfig, resume: str | None) -> str | None:
if resume != _RESUME_LATEST:
return resume
latest_path = config.checkpoint_path / "latest.pt"
if not latest_path.exists():
raise FileNotFoundError(
f"--resume was used without a path, but latest checkpoint does not exist: {latest_path}"
)
return str(latest_path)
def _train_overrides_from_args(args: argparse.Namespace) -> dict[str, Any]:
overrides: dict[str, Any] = {}
for assignment in getattr(args, "config_overrides", None) or ():
@@ -89,7 +92,10 @@ def train_command(args: argparse.Namespace) -> None:
config = _load_config(args.config)
overrides = _train_overrides_from_args(args)
config = _with_overrides(config, overrides)
resume_path = _resolve_resume_path(config, args.resume)
if args.resume:
run_dir = Path(args.resume).parent
else:
run_dir = _resolve_run_dir(config, keep=args.keep)
extra_trackers: list[RunTracker] = []
if args.wandb:
extra_trackers.append(
@@ -98,18 +104,19 @@ def train_command(args: argparse.Namespace) -> None:
name=args.wandb_name or config.run.experiment_name,
mode=args.wandb_mode,
config=config.to_dict(),
run_dir=str(config.checkpoint_path),
run_dir=str(run_dir),
tags=list(args.wandb_tag) if args.wandb_tag else None,
)
)
trainer = DeepCFRTrainer(
config,
config.rules.to_lost_cities_config(seed=config.run.seed),
run_dir=run_dir,
device=config.run.device,
extra_trackers=extra_trackers or None,
)
if resume_path:
trainer.load_checkpoint(resume_path)
if args.resume:
trainer.load_checkpoint(args.resume)
trainer.train()
@@ -203,7 +210,15 @@ def main(argv: list[str] | None = None) -> None:
train = subparsers.add_parser("train")
train.add_argument("--config")
train.add_argument("--resume", nargs="?", const=_RESUME_LATEST, default=None)
train.add_argument(
"--resume",
help="Path to a checkpoint to resume from (e.g. runs/.../latest.pt).",
)
train.add_argument(
"--keep",
action="store_true",
help="Place run under runs/ instead of runs/tmp/ (use for real experiments).",
)
train.add_argument(
"--set",
action="append",
@@ -217,16 +217,11 @@ class MemoryConfig(StrictModel):
class CheckpointConfig(StrictModel):
directory: str = "runs/deep_cfr/default"
save_every: int = 1
save_latest: bool = True
progress_interval_seconds: float = 20.0
exact_resume: bool = False
@property
def path(self) -> Path:
return Path(self.directory)
class EvaluationConfig(StrictModel):
eval_every: int = 50
@@ -278,10 +273,6 @@ class DeepCFRConfig(StrictModel):
def to_dict(self) -> dict[str, Any]:
return self.model_dump(mode="json")
@property
def checkpoint_path(self) -> Path:
return self.checkpoint.path
def config_from_dict(data: Mapping[str, Any]) -> DeepCFRConfig:
return DeepCFRConfig.model_validate(data)
@@ -157,6 +157,7 @@ class DeepCFRTrainer:
config: DeepCFRConfig | None = None,
game_config: LostCitiesConfig | None = None,
*,
run_dir: str | Path | None = None,
device: str = "cpu",
tracker: RunTracker | None = None,
extra_trackers: list[RunTracker] | None = None,
@@ -200,7 +201,7 @@ class DeepCFRTrainer:
self.strategy_memory = ReservoirMemory(self.config.memory.strategy_capacity)
self.rng = np.random.default_rng(self.config.run.seed + 101)
self.iteration = 0
self.run_dir = self.config.checkpoint_path
self.run_dir = Path(run_dir) if run_dir is not None else Path("runs/tmp/default")
self.metrics_path = self.run_dir / "metrics.jsonl"
self.progress_path = self.run_dir / "runtime_progress.json"
self.log_path = self.run_dir / "train.log"
+35 -56
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import re
from pathlib import Path
import numpy as np
import torch
@@ -14,8 +15,8 @@ from coolrl_lost_cities.games.classic.deep_cfr.benchmark import (
)
from coolrl_lost_cities.games.classic.deep_cfr.checkpoints import load_checkpoint
from coolrl_lost_cities.games.classic.deep_cfr.cli import (
_RESUME_LATEST,
_resolve_resume_path,
_kebab_slug,
_resolve_run_dir,
_train_overrides_from_args,
_with_overrides,
)
@@ -36,13 +37,13 @@ def test_deep_cfr_loads_smoke_yaml_config() -> None:
assert config.run.max_iterations == 1
assert config.network.hidden_size == 16
assert config.traversal.traversals_per_player == 1
assert config.checkpoint.directory == "runs/deep_cfr/smoke"
assert config.run.experiment_name == "smoke"
def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
config = load_config("configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml")
config = load_config("configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml")
assert config.run.experiment_name.endswith("slot_playability")
assert config.run.experiment_name.endswith("slot-playability")
assert config.run.seed == 79
assert config.run.max_iterations is None
assert config.run.max_minutes == 240
@@ -69,9 +70,6 @@ def test_deep_cfr_loads_mapped_legacy_reproduction_config() -> None:
assert config.regret_matching.all_negative_fallback == "uniform"
assert config.training_weighting.mode == "none"
assert config.checkpoint.save_every == 10
assert (
config.checkpoint.directory == "runs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability"
)
def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None:
@@ -91,7 +89,7 @@ def test_deep_cfr_train_cli_accepts_run_and_traversal_config_overrides() -> None
],
},
)()
config = load_config("configs/deep_cfr/deep_cfr_selfplay_full_depth_slot_playability.yaml")
config = load_config("configs/deep_cfr/deep-cfr-selfplay-full-depth-slot-playability.yaml")
overridden = _with_overrides(config, _train_overrides_from_args(args))
@@ -216,25 +214,18 @@ def test_deep_cfr_batched_evaluation_matches_batch_size_one() -> None:
assert np.isclose(batched["policy_entropy"], batch_one["policy_entropy"])
def test_deep_cfr_resume_latest_resolution_uses_config_checkpoint_dir(tmp_path) -> None:
config = _deep_cfr_config({"checkpoint": {"directory": str(tmp_path)}})
latest = tmp_path / "latest.pt"
latest.write_bytes(b"checkpoint")
def test_deep_cfr_resolve_run_dir_uses_keep_flag_and_kebab_slug() -> None:
config = _deep_cfr_config({"run": {"experiment_name": "Color Shared Attn v2"}})
assert _resolve_resume_path(config, _RESUME_LATEST) == str(latest)
assert _resolve_resume_path(config, "custom.pt") == "custom.pt"
assert _resolve_resume_path(config, None) is None
tmp_path = _resolve_run_dir(config, keep=False)
keep_path = _resolve_run_dir(config, keep=True)
def test_deep_cfr_resume_latest_resolution_requires_latest(tmp_path) -> None:
config = _deep_cfr_config({"checkpoint": {"directory": str(tmp_path)}})
try:
_resolve_resume_path(config, _RESUME_LATEST)
except FileNotFoundError as exc:
assert "latest checkpoint does not exist" in str(exc)
else: # pragma: no cover
raise AssertionError("expected FileNotFoundError")
assert tmp_path.parent == Path("runs/tmp")
assert keep_path.parent == Path("runs")
assert tmp_path.name.endswith("_color-shared-attn-v2")
assert keep_path.name.endswith("_color-shared-attn-v2")
assert _kebab_slug("Foo BAR_baz!!") == "foo-bar-baz"
assert _kebab_slug(" ") == "run"
def test_deep_cfr_playability_encoding_extends_input_shape() -> None:
@@ -321,13 +312,11 @@ def test_deep_cfr_trainer_forwards_metrics_to_extra_trackers(tmp_path) -> None:
"max_nodes_per_traversal": 16,
},
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": {
"directory": str(tmp_path / "extra-tracker"),
"save_every": 0,
},
"checkpoint": {"save_every": 0},
}
),
LostCitiesConfig(seed=71),
run_dir=tmp_path / "extra-tracker",
extra_trackers=[_CaptureTracker()],
)
@@ -643,14 +632,12 @@ def test_deep_cfr_trainer_saves_loads_and_evaluates_checkpoint(tmp_path) -> None
"max_nodes_per_traversal": 32,
},
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": {
"directory": str(checkpoint_dir),
"save_every": 1,
},
"checkpoint": {"save_every": 1},
"evaluation": {"eval_every": 1, "games": 2, "opponents": ("random",)},
}
),
LostCitiesConfig(seed=41),
run_dir=checkpoint_dir,
)
metrics = trainer.train()
@@ -660,13 +647,11 @@ def test_deep_cfr_trainer_saves_loads_and_evaluates_checkpoint(tmp_path) -> None
{
"run": {"seed": 41},
"network": {"hidden_size": 16},
"checkpoint": {
"directory": str(checkpoint_dir),
"save_every": 0,
},
"checkpoint": {"save_every": 0},
}
),
LostCitiesConfig(seed=41),
run_dir=checkpoint_dir,
)
restored.load_checkpoint(latest)
@@ -701,13 +686,11 @@ def test_deep_cfr_trainer_always_saves_latest_checkpoint(tmp_path) -> None:
"run": {"max_iterations": 1, "seed": 42},
"network": {"hidden_size": 16},
"traversal": {"traversals_per_player": 1, "max_depth": 1},
"checkpoint": {
"directory": str(checkpoint_dir),
"save_every": 10,
},
"checkpoint": {"save_every": 10},
}
),
LostCitiesConfig(seed=42),
run_dir=checkpoint_dir,
)
trainer.train()
@@ -724,20 +707,22 @@ def test_deep_cfr_exact_resume_is_explicitly_not_implemented(tmp_path) -> None:
"run": {"max_iterations": 1, "seed": 44},
"network": {"hidden_size": 16},
"traversal": {"traversals_per_player": 1, "max_depth": 1},
"checkpoint": {"directory": str(checkpoint_dir), "save_every": 1},
"checkpoint": {"save_every": 1},
}
),
LostCitiesConfig(seed=44),
run_dir=checkpoint_dir,
)
trainer.train()
exact = DeepCFRTrainer(
_deep_cfr_config(
{
"network": {"hidden_size": 16},
"checkpoint": {"directory": str(checkpoint_dir), "exact_resume": True},
"checkpoint": {"exact_resume": True},
}
),
LostCitiesConfig(seed=44),
run_dir=checkpoint_dir,
)
try:
@@ -763,13 +748,11 @@ def test_deep_cfr_trainer_multiprocessing_smoke_run(tmp_path) -> None:
"progress_every_traversals": 1,
},
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": {
"directory": str(tmp_path / "mp"),
"save_every": 0,
},
"checkpoint": {"save_every": 0},
}
),
LostCitiesConfig(seed=43),
run_dir=tmp_path / "mp",
)
metrics = trainer.train()
@@ -827,13 +810,11 @@ def test_deep_cfr_self_play_league_records_snapshots(tmp_path) -> None:
"anchor_probability": 1.0,
},
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": {
"directory": str(tmp_path / "league"),
"save_every": 0,
},
"checkpoint": {"save_every": 0},
}
),
LostCitiesConfig(seed=53),
run_dir=tmp_path / "league",
)
metrics = trainer.train()
@@ -864,13 +845,11 @@ def test_deep_cfr_weighted_self_play_league_uses_snapshot_bucket(tmp_path) -> No
"recent_window": 1,
},
"optimization": {"advantage_batch_size": 2, "strategy_batch_size": 2},
"checkpoint": {
"directory": str(tmp_path / "weighted-league"),
"save_every": 0,
},
"checkpoint": {"save_every": 0},
}
),
LostCitiesConfig(seed=59),
run_dir=tmp_path / "weighted-league",
)
metrics = trainer.train()