scripts/librarian_check_citations.py extracts file:line references from
inline-code spans across docs/**/*.md and verifies each path exists
(and, if cited with a line number, is within range). Skips
docs/archive/ and docs/plans/archive/ which are read-only by policy.
Caught one real drift in docs/research/optimization_sequencing.md: the
note pointed at docs/plans/amp_trainer.md, which had moved into
docs/plans/archive/.
scripts/librarian-ignore.txt holds fnmatch globs for citations that are
intentionally future-tense (planned files described in the plan docs
themselves). Used sparingly so the checker stays useful as a drift
signal.
Updates docs/plans/librarian.md Progress + Next Step. Next concrete
step is a thin scripts/librarian.sh orchestrator over both checkers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the full librarian design (two-layer architecture, three-stage
pipeline, vendor-agnostic via LIBRARIAN_LLM env, propose-only / no
auto-apply) in docs/plans/librarian.md. Lands the first concrete Stage 1
piece: scripts/librarian_check_links.py, a lychee --offline wrapper
ported from ~/dev/coolrl/src/coolrl/dev/check_doc_links.py.
Also moves the librarian prompt from .claude/agents/ (Claude Code only)
to scripts/librarian-prompt.md so any CLI can load it as a system
prompt later. Fixes one stale README link the new checker caught:
docs/classic-port-notes.md → docs/archive/classic-port-notes.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an experiment-only explicit-continuation traversal prototype that batches policy requests, records CPU/CUDA parity results, and updates the Option B plan with Phase 0/1 evidence.
Co-Authored-By: Codex <codex@openai.com>
Archive implemented AMP, Option A inference-server, and Cython heuristic plans. Add the active Option B interleaved traversal plan and update model-size/torch.compile plans to reflect the current traversal scheduling conclusion.
Co-Authored-By: Codex <codex@openai.com>
Add a microbench that separates game/encoding overhead, single-request policy boundary overhead, and batched PyTorch forward lower bounds. Record CPU/CUDA results and link the finding from the Julia port evaluation.
Co-Authored-By: Codex <codex@openai.com>
Captures the dependency graph between pending levers (model size,
Option B, AMP, compile, TensorRT, Option A re-enable, Julia port) and
the rule that infrastructure optimization precedes the model-size
experiment because every future training run benefits from the
infrastructure speedup, not just the one keystone experiment.
ideas.md gets a third Active Research Threads pointer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Decision-in-advance for criteria 3 (multi-thread scaling), 4 (Flux/CUDA
MLP forward), and 5 (real-game-state slice). Each criterion lists PASS
/ PARTIAL / FAIL bands with concrete numerical thresholds, plus a
decision rule that maps {3, 4, 5} outcomes to a single action: port,
hybrid (Julia traversal + PyTorch networks), or stay on Python/Cython
and pursue Option B instead.
Cost-of-being-wrong asymmetry stated explicitly: port is months,
staying is zero work, so the GO bar is deliberately above 50% and the
STAY bar is permissive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
experiments/julia_cfr_toy/ ports a synthetic CFR external-sampling
traversal to both Julia and Cython for direct head-to-head
measurement of the actual project hot-path pattern (recursive
tree + mutable regret state + branch-heavy legal-action logic).
Headline result (2026-05-07, single run): Julia ~1.9× faster than
Cython on this pattern, 0 MB allocation, 0% GC time. Root regret
parity ε ≤ 1e-9. The GC-pause concern that was the main argument
against Julia adoption did not materialize. Cython's 21.3 MB
allocation suggests its implementation can be tightened, so the
honest gap window is roughly 1.3×–1.9×.
Multi-thread scaling (bench_cfr_threaded.jl): 2.44× wall-clock at 8T
but only 31% efficiency — inconclusive, likely a toy-size artifact
(per-thread workload too small to amortize dispatch). A heavier
per-thread workload sweep is the remaining decisive test.
docs/research/julia_port_evaluation.md captures this evidence
alongside the earlier safe-heuristic single-thread parity result
and lists the remaining decision criteria (multi-thread scaling with
heavier workload, Flux.jl+CUDA.jl coverage, real-game-state slice).
Do not commit to porting until multi-thread scaling is conclusively
settled.
ideas.md gets a second Active Research Threads pointer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mechanism-level analysis of the slot_aware_playability iter 240
plateau (opened_colors 4.95+, bad_open_rate 88-92%, calibration
gap 6-9 → 2-4). Captures four expert consultations with diagnostic
hypotheses, intervention catalog (architectural / training-dynamics
/ game-specific), measurement plan, and a comparison table across
the four sources.
Key new directions surfaced:
- Current vs average vs league policy separation (Deep CFR average
strategy is the convergence target, not advantage current).
- All-negative fallback as Deep CFR ablation lever.
- Empirical r̃ partitioning by action class.
- Tabular Lost Cities oracle as a clean test of "is 5-color the
game-theoretic answer or an approximation artifact".
- Entry-gate target defined from traversal counterfactual values
instead of heuristic labels.
ideas.md gets an "Active Research Threads" pointer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OpenSpiel's Deep CFR records strategy samples at opponent nodes during
the traverser's tree walk; storing on traverser nodes under external
sampling drops the ρ_p reach factor and biases the average-policy
estimate. Reject that config combination at load time and add a research
note deriving why outcome sampling is unaffected while external is not.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five notes covering outcome-sampling target correctness, package
architecture, v0 feature-parity vs legacy, opponent-policy network
divergence, and regret-matching fallback audit. Four are derived from
archive sources (cited via Source: lines); outcome-sampling-target is
a fresh write-up and serves as the style template.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds torch.autocast(fp16) + GradScaler around _train_advantage and
_train_strategy when run.use_amp=true and device=cuda. CPU/non-CUDA
falls back to fp32 no-op. Mitigations:
- scaler.unscale_(optimizer) before grad_clip.
- nonfinite-loss guard skips overflowing batches and counts them.
- diff.float().square() in advantage loss to avoid fp16 overflow.
- strategy mask/log_softmax kept in fp32.
New metrics: amp/grad_scale, amp/nonfinite_loss_count.
Tests: AMP CUDA smoke + CPU fallback in test_deep_cfr_trainer.py.
Bench: scripts/bench_amp_trainer.py micro-benches train phases under
synthetic replay memory. smoke.yaml result is fp32 3.22ms / AMP 3.92ms
(0.82×, regression). 100-iter A/B on default.yaml deliberately
skipped: smoke regression mirrors the 2026-05-07 torch.compile
regression dynamic (dispatch overhead > kernel benefit at this model
size) and re-confirming on the same size adds no information.
Default stays run.use_amp: false. Re-enable trigger documented in
docs/performance.md: hidden_size >= 1024 or num_layers >= 6, then run
the bench script + 100-iter A/B before flipping default.
The experiment finds a model size where (a) learning-curve gains
justify compute, and (b) forward time is large enough to amortize
AMP/compile/TensorRT overhead. Outcome gates re-enabling those four
deferred optimizations and Option A.
Tested grid: hidden={512,768,1024,1536} × layers={3,4,6,8} subset.
200 iterations per config on home (RTX 3090), single seed initially,
second seed for boundary configs. Eval cadence held at default.
Decision tree included for: success → recommend new default and
trigger downstream plans; null → document and stay; expensive-but-
better → defer until AMP/compile/TRT land.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
performance.md additions:
- Batched Traversal Inference design decision (A vs B vs C with
rationale).
- Option A bench result and structural ceiling (realized batch ~7.2,
IPC overhead exceeds GPU gain at small model size).
- Post-A optimization calculus: why compile/TensorRT remain
iter-neutral today and become meaningful only after model growth
and/or denser eval. Sequencing matters; do not retest these on the
current small model.
- Free-threaded Python (3.13t/3.14t) note: cleanest endpoint in
principle, but PyTorch maturity + Cython nogil audit cost block
near-term adoption.
docs/plans/ (4 plan documents for Codex execution):
- batched_traversal_inference_server.md (executed; deferred).
- amp_trainer.md.
- torch_compile.md.
- cython_safe_heuristic_bots.md (executed; first-pass landed).
docs/reports/ (3 cost reports):
- cost_pytorch_free_threaded_2026-05-07.md: WAIT 3-6 months;
PyTorch wheels exist but our Cython is the gating cost.
- cost_cython_nogil_audit_2026-05-07.md: medium effort, traversal.pyx
carries 90% of blockers; Steps 1-3 (cfr_math/encoding nogil
keywords, TraversalStats cdef class) are safe and cheap, Steps
4-6 wait for triggers.
- cost_pytorch_cuda_multithread_2026-05-07.md: risky;
optimizer.step / load_state_dict race silently with concurrent
forward; per-thread default streams unset means naive threading
serializes on default stream anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Measured DeepCFRMLP forward at bs={1,4,16,64,256,1024} on RTX 3090.
Per-state cost drops 232× from bs=1 (80 µs) to bs=256 (0.34 µs) while
per-call latency stays near 90 µs through bs=256. Policy-call supply
from a real run is ~368 states per traversal and ~200k per iteration,
well above the bs=64–256 plateau, so batched inference is not
supply-limited. GPU forward is not the limiter once batching exists.
Verdict: Optimization Priorities #5 (batched traversal inference) is
worth pursuing. End-to-end gain will still be bounded by encoding and
worker-GPU coordination overhead.
- scripts/profile_gpu_forward.py: standalone profiling script
- docs/performance.md: new "GPU forward profiling for batched traversal"
experiment section with table, supply estimate, and verdict
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Make the link from the failed torch.compile experiment to Optimization
Priorities #5 explicit, so a future revisit happens at the right time
(once compile is on the dominant phase, not just trainer optimization
steps).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wrapping trainer networks with torch.compile produced a 4.8% regression
in iteration time on default.yaml (17.93s → 18.79s). Two causes: (1)
the dominant phase is CPU traversal which bypasses the compiled
wrapper, (2) DeepCFRMLP is too small for compile dispatch overhead to
pay back. Implementation kept on experiments/torch-compile for future
revisits when the trainer model or inference path changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
같은 hyperparameter (512x3, 2x updates, argmax_tiebreak)에 opponent_policy
만 self_play_league로 다른 run과의 iteration별 비교 표 추가. iter 350
시점에서 Random WR 38%p 격차 (network 34% vs league 72%) 확인 — 발산
가설을 실증.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
opponent_policy=network 설정으로 1000-iter 실험 두 개 (512x3, 1024x4)를
돌린 결과 두 실험 모두 policy collapse가 발생함을 확인. 큰 capacity는
plateau를 늘리지만 발산 자체를 막지 못함. 향후 학습은 self_play_league
기본값을 유지할 것을 권고.
- 실험에 사용한 config 두 개 추가
- 발견 분석 문서 추가 (원인, 비교, 권고)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
맥락: 클래식 포트를 Python/Cython 구현과 pygame GUI 중심으로 정리한다.
변경: Rust 크레이트, proto 계약, Rust backend shim, 패리티 테스트를 삭제하고 문서와 backend factory를 Python 전용으로 갱신했다.
확인: uv run pre-commit run --all-files; uv run pytest tests/games/classic; uv run lost-cities-classic-gui --help
coolguyandClaude Opus 4.7