coorl-lost-cities/AGENTS.md

AGENTS.md

This repository is managed with uv. Use uv run ... for commands so the project environment and Cython extensions are built/loaded consistently.

Project Layout

• src/coolrl_lost_cities/games/classic/game.pyx: Cython Lost Cities engine.
• src/coolrl_lost_cities/games/classic/deep_cfr/: Deep CFR training, traversal, evaluation, analysis, and CLI code.
• configs/deep_cfr/: active Deep CFR YAML configs (kebab-case filenames). Currently holds two:
  • default.yaml: the canonical "best-known" baseline. Start here, then override fields via --set for experiments/ablations.
  • smoke.yaml: 1-iter sanity check for the training loop.
• configs/archive/: retired/historical configs. Don't modify; reference if you need to reproduce an old run.
• runs/: generated training runs. Gitignored, may be a symlink to larger storage. Layout:
  • runs/archive/: past runs. Do not modify or delete.
  • runs/tmp/: smoke, tests, throwaway. Free to rm -rf anytime.
  • runs/<YYYY-MM-DD_HHMMSS>_<kebab-name>/: real experiments (flat). Promote a runs/<...> directory to runs/archive/ with a manual mv once analysis is complete.
• docs/: profiling notes, migration notes, and experiment documentation.

Core Commands

Run lint:

uv run ruff check .

Run all tests:

uv run pytest -q

Run focused Deep CFR tests:

uv run pytest -q tests/games/classic/test_deep_cfr_trainer.py

Run the CLI through the console script:

uv run lost-cities-deep-cfr --help

Equivalent module form:

uv run python -m coolrl_lost_cities.games.classic.deep_cfr.cli --help

Deep CFR Training

The CLI auto-derives the run directory from run.experiment_name plus a timestamp. By default runs land under runs/tmp/; pass --keep for a real experiment that should live under runs/.

Smoke / throwaway run (lands in runs/tmp/):

uv run lost-cities-deep-cfr train --config configs/deep_cfr/smoke.yaml
# → runs/tmp/<YYYY-MM-DD_HHMMSS>_smoke/

Real experiment (lands in runs/):

uv run lost-cities-deep-cfr train \
  --config configs/deep_cfr/default.yaml \
  --keep
# → runs/<YYYY-MM-DD_HHMMSS>_deep-cfr-default/

Variant / ablation (override one field; keep slug informative):

uv run lost-cities-deep-cfr train \
  --config configs/deep_cfr/default.yaml \
  --keep \
  --set training_weighting.mode=none \
  --set run.experiment_name=ablation-no-lcfr
# → runs/<YYYY-MM-DD_HHMMSS>_ablation-no-lcfr/

Short fixed-iteration run:

uv run lost-cities-deep-cfr train \
  --config configs/deep_cfr/default.yaml \
  --set run.max_iterations=100 \
  --set checkpoint.save_every=0

Resume (path required, no shortcut):

uv run lost-cities-deep-cfr train \
  --config configs/deep_cfr/default.yaml \
  --resume runs/<YYYY-MM-DD_HHMMSS>_<slug>/latest.pt

When --resume is given, the trainer reuses the resumed checkpoint's parent directory; no new timestamped folder is created.

Useful train controls:

• --keep: real experiment, write under runs/ (default is runs/tmp/).
• --resume PATH: resume from a specific checkpoint. PATH is required.
• --set PATH=VALUE: override config fields. Repeatable, parses values as YAML (e.g. --set traversal.num_workers=4, --set run.max_minutes=null).

Common --set overrides:

• --set run.device=cuda: set the trainer device.
• --set run.experiment_name=foo-v2: change the slug used in the run dir name (kebab-case).
• --set checkpoint.exact_resume=true: require checkpoint config compatibility.
• --set checkpoint.save_latest=false --set checkpoint.save_every=0: disable checkpoint writes.
• --set checkpoint.save_every=0: keep only latest.pt (no archives).
• --set checkpoint.save_every=N: archive every N iterations.

Naming Conventions

• Directory names, run dirs, config filenames, experiment_name values: kebab-case (deep-cfr-color-shared-512x3.yaml, runs/2026-05-08_103045_color-attn-v2/).
• YAML keys, Python identifiers, config field names: snake_case (unchanged: hidden_size, traversals_per_player, experiment_name).
• The CLI converts run.experiment_name to a kebab slug when building the run directory, so values may contain spaces or mixed case.

Long Runs

Run long jobs in a real tmux session so the user can attach and stop them. Do not rely on Codex command sessions for long user-observable training runs.

Start a long unbounded run:

tmux new-session -s coolrl-deepcfr-unbounded \
  -c /home/coolguy/dev/coolrl-lost-cities \
  'uv run lost-cities-deep-cfr train \
    --config configs/deep_cfr/default.yaml \
    --set run.max_iterations=null \
    --set run.max_minutes=null \
    --keep'

Attach later:

tmux attach -t coolrl-deepcfr-unbounded

Detach without stopping:

Ctrl+B, D

Stop training:

Ctrl+C

Follow logs from another terminal:

tail -f runs/<YYYY-MM-DD_HHMMSS>_<slug>/train.log

The unbounded config intentionally has:

run:
  max_iterations: null
  max_minutes: null
checkpoint:
  save_every: 100

latest.pt is updated continuously; archive checkpoints are written every 100 iterations. If disk is tight, set --set checkpoint.save_every=0 (keep only latest.pt) or increase save_every.

Evaluation And Analysis

Evaluate a checkpoint:

uv run lost-cities-deep-cfr eval \
  --checkpoint runs/<run-dir>/latest.pt \
  --opponent random \
  --games 100 \
  --device cpu

Save evaluation game records:

uv run lost-cities-deep-cfr eval \
  --checkpoint runs/<run-dir>/latest.pt \
  --opponent random \
  --games 100 \
  --device cpu \
  --save-games runs/<run-dir>/eval_random_games.json

Generate analysis plots from metrics.jsonl:

uv run lost-cities-deep-cfr analyze \
  --run runs/<run-dir>

Write plots to a separate directory:

uv run lost-cities-deep-cfr analyze \
  --run runs/<run-dir> \
  --output-dir runs/<run-dir>/analysis

The analyzer reads metrics.jsonl and writes PNG files grouped by diagnostic section. Opponents are compared within each plot using fixed colors. The lost-cities-deep-cfr analyze subcommand uses the analyzer default smoothing window, currently 1 iteration (no smoothing), and supports --max-iteration.

For smoothing controls, run the analyzer module directly:

uv run python -m coolrl_lost_cities.games.classic.deep_cfr.analyze \
  --run runs/<run-dir> \
  --smoothing-window 5

Use --no-smoothing to force no moving average.

Current output files:

• analysis_01_loss.png
• analysis_02_match.png
• analysis_03_action.png
• analysis_04_gameflow.png
• analysis_05_open_quality.png
• analysis_06_expedition_outcomes.png
• analysis_07_calibration.png
• analysis_08_traversal.png
• analysis_09_selectivity.png
• analysis_final_eval_summary.png

Runtime Artifacts

Each training run writes:

• metrics.jsonl: structured metrics, one completed iteration per line.
• train.log: human-readable timestamped logs.
• runtime_progress.json: latest progress snapshot.
• latest.pt: latest checkpoint.
• iteration_*.pt: archive checkpoints when enabled.
• config.json: resolved config for the run.

If a run is stopped mid-iteration, the in-progress iteration may not appear in metrics.jsonl. Analyze the latest completed metric row.

Weights & Biases (Optional)

Metrics can be mirrored to W&B. wandb is an optional extra; default installs and runs do not require it.

Install:

uv sync --extra wandb

Run with W&B:

# Offline: no login, writes to <run_dir>/wandb/offline-run-*/
uv run lost-cities-deep-cfr train --config <...> --wandb --wandb-mode offline

# Online: requires `uv run wandb login` once, then real-time upload
uv run lost-cities-deep-cfr train --config <...> --wandb

W&B data is stored per run at <run_dir>/wandb/, not at a global runs/wandb/. Each training run gets its own subfolder, so moving or deleting a run directory carries its W&B data along with it.

Sync offline runs to wandb.ai later:

wandb sync runs/<run-dir>/wandb/offline-run-*

Flags:

• --wandb: enable W&B mirroring.
• --wandb-project <name>: defaults to coolrl-lost-cities.
• --wandb-name <name>: W&B run name; defaults to run.experiment_name.
• --wandb-mode {online,offline,disabled}: default online.
• --wandb-tag <tag>: tag the run; repeatable.

W&B is purely additive — metrics.jsonl remains the source of truth, and analyze reads metrics.jsonl, not W&B. Disabling W&B never breaks training, resume, or analysis.

Notes and tags

Use --wandb-notes for the run's purpose (free-form prose) and --wandb-tag for categories you might filter on later (short kebab-case keywords, repeatable). Otherwise use them however you like. Just avoid:

• Tags that duplicate config (lr-1e-4, traversal-280) — W&B already indexes config fields.
• Tags that are unique per run (test-1, 2026-05-07) — that's the run name and timestamp's job.
• Tag-as-sentence (tested-bigger-traversal-with-lcfr) — that belongs in --wandb-notes.

Notes length: 3–5 lines, commit-message-body length. Should answer why (hypothesis), what (key config delta), and baseline (run/iter to compare against). Long analyses go in docs/ and are linked from notes; don't paste them in.

Comparing two runs

Default is sequential, single seed. Run baseline first, then the treatment with exactly one config change, both with the same run.seed. Tag both with a shared hypothesis tag (e.g. --wandb-tag lr-bump) so they show up together in W&B's Compare Runs view.

Do not run multiple seeds per condition unless explicitly asked — that doubles or quintuples wall-clock and isn't the default protocol. Single-seed comparison is enough to surface a signal; multi-seed is a follow-up to confirm it.

Do not run two trainings in parallel on the same GPU — VRAM/SM contention slows both unevenly and breaks the comparison.

Docs & Experiment Workflow

Where to write what

기록 내용	쓸 곳
진행 중인 계획/가설	docs/plans/<topic>.md (1 주제 1 파일)
끝난 계획	docs/plans/archive/ (수동 mv)
날짜 박힌 실험 기록	docs/archive/<name>-YYYY-MM-DD.md (immutable)
항구적 알고리즘 노트	docs/research/<name>.md (Last verified: 헤더)
비용/프로파일	docs/reports/<name>-YYYY-MM-DD.md (dated)
스크래치 / 연구 스레드 인덱스	ideas.md

Rules

• docs/archive/, runs/archive/는 read-only.
• archive 본문 복붙 금지 — 대신 Source: 링크 + distill.
• file.py:NN 인용은 작성 시점에 rg로 검증.
• 한 주제 한 파일 — foo-v2.md 만들지 말 것.
• 파일당 ~500줄 soft cap.

Notes For Future Agents

• Prefer rg/rg --files for search.
• Use apply_patch for manual edits.
• Do not commit generated run artifacts from runs/.
• Cython-generated .c files are gitignored; edit .pyx/.pxd sources.
• Before committing, run uv run ruff check . and at least the relevant pytest subset. For Deep CFR changes, run uv run pytest -q tests/games/classic/test_deep_cfr_trainer.py.

Git Branching Policy (READ THIS)

DO NOT create new git branches unless the user explicitly asks for one in the current task. Work on whatever branch is currently checked out (default main). This is a hard rule — no exceptions for "safety", "isolation", "experiments", "work-in-progress", or any other self-justified reason.

Why this rule exists:

• This project intentionally develops on main with frequent small commits.
• Auto-created branches like experiments/foo, feature/bar, wip/baz fragment review, hide work from the user, and require manual cleanup.
• The user has not authorized branch creation as a default behavior. If they want a branch, they will say so explicitly ("make a branch", "PR this", "isolate this in a branch", etc.).

What you MUST do instead:

• Make commits directly on the currently checked-out branch.
• If you think a branch is justified, stop and ask the user first — do not preemptively create one.
• If a tool or subagent invocation auto-suggests creating a branch (e.g. PR-style workflows), refuse the branch creation step and commit to the current branch.
• If you find yourself already on a non-default branch you didn't expect, stop and ask the user — do not switch, do not create a new one, do not reset.

Forbidden without explicit user instruction:

• git checkout -b <name>
• git switch -c <name>
• git branch <name>
• gh pr create from an auto-created branch
• Any worktree creation that implicitly creates a new branch

This rule applies to the main agent and to every subagent or tool the main agent invokes. Pass it through in subagent prompts when delegating git-touching work.