scripts/librarian_promote.py gains a --accept flag that copies the
generated draft to the suggested docs/research/ target in the same
invocation. This is explicit per-invocation opt-in, not auto-apply:
the operator types --accept knowingly, the cp is still a deliberate
acceptance decision, just expressed in one command instead of two.
docs/research/option-a-bench-result.md is the first promoted note,
generated by gemini from
docs/archive/option-a-bench-result-2026-05-07.md and accepted
verbatim. Spot-check verified that cited file:line locations match
current source (traversal.pyx:473-475 and
inference_server.py:226-228 carry the cited code), the Last-verified
header reflects today's date and HEAD, and the durable conclusion
(sync-blocking policy boundary as the structural ceiling, not IPC
plumbing) is preserved.
This closes the end-to-end loop the librarian was designed for:
oversize check surfaced docs/performance.md, the routing pass split
durable analysis into the archive entry, the promote dispatcher
turned the archive entry into a research draft, and the human
accept step landed it as a tracked research note. Took one LLM call.
Removes the now-resolved ignore-list entry for
docs/research/option-a-bench-result.md (the file exists; the
forward reference is real).
scripts/librarian.sh exits 0 against the working tree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/librarian_promote.py is the first Stage 2 piece: a
vendor-agnostic LLM dispatcher that drafts a docs/research/ note
from a given docs/archive/ entry. It assembles the prompt by
stitching scripts/librarian-prompt.md (system) onto the archive
body with a "draft a research note per the rules above" task
instruction, then shells out to the CLI selected by LIBRARIAN_LLM
({claude|codex|gemini}; default claude). The LLM's stdout is
captured to runs/tmp/librarian-promote-<timestamp>-draft.md for
human review — the script never writes into docs/research/ itself.
Refusal cases:
- path not under docs/archive/
- target docs/research/<stem>.md already exists (after stripping any
-YYYY-MM-DD suffix)
- archive missing
If the LLM judges the archive non-promotable, it is instructed to
return a single SKIP: <reason> line instead of fabricating a draft.
--show-prompt prints the assembled prompt without invoking the LLM,
useful for inspecting what would be sent.
Plan updated: Stage 2 v1 marked complete; remaining Stage 2 work
(MEMORY drift fixup, duplicate-merge, survey mode) catalogued.
Next concrete step is a smoke test against one real archive entry.
Adds one ignore-list entry for docs/research/option-a-bench-result.md
which appears in the plan as a hypothetical accept target.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
docs/performance.md grew to 914 lines because dated experiments and
design analyses kept getting appended instead of routed to
docs/archive/ and docs/research/ as AGENTS.md prescribes. The
oversize check from librarian Stage 1 surfaced the file; this commit
acts on that finding by extracting the parts that belong elsewhere
and trimming the source to a focused current-state reference.
Extracts (verbatim from the original prose, with cross-link headers
and a brief routing note added at top):
- docs/archive/deep-cfr-performance-experiments-2026-05-07.md
bundles torch.compile (regression), AMP (regression), GPU forward
profiling (decision support), and Option B interleaved traversal
(pass) — same date, same theme.
- docs/research/batched-traversal-inference-decision.md captures the
durable A vs B vs C rationale with a closing "Outcome" pointer to
the post-bench archive doc.
- docs/archive/post-a-optimization-calculus-2026-05-07.md preserves
the forward-looking sequencing recorded pre-bench.
- docs/archive/option-a-bench-result-2026-05-07.md preserves the
regression diagnosis and re-enable criteria.
docs/performance.md is now 345 lines, holds sections 1–9 (current
runtime / bottleneck / device / AMP status / batching / eval /
TensorRT / priorities), and ends with a "See Also" linking the four
extracts.
Also reworded the AGENTS.md soft-cap rule from a bare "~500-line
soft cap" to clarify the intent: the cap is a *routing trigger* (is
content piling up that should live in archive/research?), not a
split mandate. Reduces the risk of future agents shredding a useful
doc just to satisfy a number.
scripts/librarian.sh now exits 0 against the working tree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the last two deterministic Stage 1 checks and wires them into
scripts/librarian.sh:
- librarian_check_stale_plans.py: per docs/plans/*.md, uses
`git log -1 --format=%cs` to get the last commit date and flags
plans untouched in 60+ days. Plans should be either active or
archived; long silence is drift.
- librarian_check_memory_drift.py: validates the user-memory dir
(~/.claude/projects/<slug>/memory). Each MEMORY.md index line
must point at a real file with frontmatter (name, description,
type ∈ {user, feedback, project, reference}); no orphaned memory
files. Memory dir is derived from repo root for portability.
Both run clean against current state.
Stage 1 declared complete. Promotable-archive and duplicate-prose
detection are moved to Stage 2 in the plan because both need LLM
judgment to avoid false positives — not pattern matching.
Next concrete step is the open Stage 1 finding from yesterday:
docs/performance.md at 914 lines, due for a split into sub-topic
notes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Third Stage 1 piece: scripts/librarian_check_oversize.py walks
non-archive markdown files and flags any over the 500-line soft cap
declared in AGENTS.md. Wired into scripts/librarian.sh.
Caught one real finding on first run: docs/performance.md at 914
lines. Splitting it into sub-topic notes is a separate cleanup task
— surfaced for the user, not auto-applied.
Updates docs/plans/librarian.md Progress + sets the next concrete
step to a stale-plan checker.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single entry point that runs every Stage 1 check in order and
aggregates exit codes — both checks always run so users see all
findings in one pass. Two checks wired up today (lychee link
integrity, file:line citations), more land incrementally as Stage 1
grows.
Removes scripts/librarian.sh from the ignore list now that the file
exists. Updates docs/plans/librarian.md Progress + Next Step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/librarian_check_citations.py extracts file:line references from
inline-code spans across docs/**/*.md and verifies each path exists
(and, if cited with a line number, is within range). Skips
docs/archive/ and docs/plans/archive/ which are read-only by policy.
Caught one real drift in docs/research/optimization_sequencing.md: the
note pointed at docs/plans/amp_trainer.md, which had moved into
docs/plans/archive/.
scripts/librarian-ignore.txt holds fnmatch globs for citations that are
intentionally future-tense (planned files described in the plan docs
themselves). Used sparingly so the checker stays useful as a drift
signal.
Updates docs/plans/librarian.md Progress + Next Step. Next concrete
step is a thin scripts/librarian.sh orchestrator over both checkers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the full librarian design (two-layer architecture, three-stage
pipeline, vendor-agnostic via LIBRARIAN_LLM env, propose-only / no
auto-apply) in docs/plans/librarian.md. Lands the first concrete Stage 1
piece: scripts/librarian_check_links.py, a lychee --offline wrapper
ported from ~/dev/coolrl/src/coolrl/dev/check_doc_links.py.
Also moves the librarian prompt from .claude/agents/ (Claude Code only)
to scripts/librarian-prompt.md so any CLI can load it as a system
prompt later. Fixes one stale README link the new checker caught:
docs/classic-port-notes.md → docs/archive/classic-port-notes.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an experiment-only explicit-continuation traversal prototype that batches policy requests, records CPU/CUDA parity results, and updates the Option B plan with Phase 0/1 evidence.
Co-Authored-By: Codex <codex@openai.com>
Archive implemented AMP, Option A inference-server, and Cython heuristic plans. Add the active Option B interleaved traversal plan and update model-size/torch.compile plans to reflect the current traversal scheduling conclusion.
Co-Authored-By: Codex <codex@openai.com>
Add a microbench that separates game/encoding overhead, single-request policy boundary overhead, and batched PyTorch forward lower bounds. Record CPU/CUDA results and link the finding from the Julia port evaluation.
Co-Authored-By: Codex <codex@openai.com>
Captures the dependency graph between pending levers (model size,
Option B, AMP, compile, TensorRT, Option A re-enable, Julia port) and
the rule that infrastructure optimization precedes the model-size
experiment because every future training run benefits from the
infrastructure speedup, not just the one keystone experiment.
ideas.md gets a third Active Research Threads pointer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Decision-in-advance for criteria 3 (multi-thread scaling), 4 (Flux/CUDA
MLP forward), and 5 (real-game-state slice). Each criterion lists PASS
/ PARTIAL / FAIL bands with concrete numerical thresholds, plus a
decision rule that maps {3, 4, 5} outcomes to a single action: port,
hybrid (Julia traversal + PyTorch networks), or stay on Python/Cython
and pursue Option B instead.
Cost-of-being-wrong asymmetry stated explicitly: port is months,
staying is zero work, so the GO bar is deliberately above 50% and the
STAY bar is permissive.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
experiments/julia_cfr_toy/ ports a synthetic CFR external-sampling
traversal to both Julia and Cython for direct head-to-head
measurement of the actual project hot-path pattern (recursive
tree + mutable regret state + branch-heavy legal-action logic).
Headline result (2026-05-07, single run): Julia ~1.9× faster than
Cython on this pattern, 0 MB allocation, 0% GC time. Root regret
parity ε ≤ 1e-9. The GC-pause concern that was the main argument
against Julia adoption did not materialize. Cython's 21.3 MB
allocation suggests its implementation can be tightened, so the
honest gap window is roughly 1.3×–1.9×.
Multi-thread scaling (bench_cfr_threaded.jl): 2.44× wall-clock at 8T
but only 31% efficiency — inconclusive, likely a toy-size artifact
(per-thread workload too small to amortize dispatch). A heavier
per-thread workload sweep is the remaining decisive test.
docs/research/julia_port_evaluation.md captures this evidence
alongside the earlier safe-heuristic single-thread parity result
and lists the remaining decision criteria (multi-thread scaling with
heavier workload, Flux.jl+CUDA.jl coverage, real-game-state slice).
Do not commit to porting until multi-thread scaling is conclusively
settled.
ideas.md gets a second Active Research Threads pointer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mechanism-level analysis of the slot_aware_playability iter 240
plateau (opened_colors 4.95+, bad_open_rate 88-92%, calibration
gap 6-9 → 2-4). Captures four expert consultations with diagnostic
hypotheses, intervention catalog (architectural / training-dynamics
/ game-specific), measurement plan, and a comparison table across
the four sources.
Key new directions surfaced:
- Current vs average vs league policy separation (Deep CFR average
strategy is the convergence target, not advantage current).
- All-negative fallback as Deep CFR ablation lever.
- Empirical r̃ partitioning by action class.
- Tabular Lost Cities oracle as a clean test of "is 5-color the
game-theoretic answer or an approximation artifact".
- Entry-gate target defined from traversal counterfactual values
instead of heuristic labels.
ideas.md gets an "Active Research Threads" pointer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OpenSpiel's Deep CFR records strategy samples at opponent nodes during
the traverser's tree walk; storing on traverser nodes under external
sampling drops the ρ_p reach factor and biases the average-policy
estimate. Reject that config combination at load time and add a research
note deriving why outcome sampling is unaffected while external is not.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five notes covering outcome-sampling target correctness, package
architecture, v0 feature-parity vs legacy, opponent-policy network
divergence, and regret-matching fallback audit. Four are derived from
archive sources (cited via Source: lines); outcome-sampling-target is
a fresh write-up and serves as the style template.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds torch.autocast(fp16) + GradScaler around _train_advantage and
_train_strategy when run.use_amp=true and device=cuda. CPU/non-CUDA
falls back to fp32 no-op. Mitigations:
- scaler.unscale_(optimizer) before grad_clip.
- nonfinite-loss guard skips overflowing batches and counts them.
- diff.float().square() in advantage loss to avoid fp16 overflow.
- strategy mask/log_softmax kept in fp32.
New metrics: amp/grad_scale, amp/nonfinite_loss_count.
Tests: AMP CUDA smoke + CPU fallback in test_deep_cfr_trainer.py.
Bench: scripts/bench_amp_trainer.py micro-benches train phases under
synthetic replay memory. smoke.yaml result is fp32 3.22ms / AMP 3.92ms
(0.82×, regression). 100-iter A/B on default.yaml deliberately
skipped: smoke regression mirrors the 2026-05-07 torch.compile
regression dynamic (dispatch overhead > kernel benefit at this model
size) and re-confirming on the same size adds no information.
Default stays run.use_amp: false. Re-enable trigger documented in
docs/performance.md: hidden_size >= 1024 or num_layers >= 6, then run
the bench script + 100-iter A/B before flipping default.
The experiment finds a model size where (a) learning-curve gains
justify compute, and (b) forward time is large enough to amortize
AMP/compile/TensorRT overhead. Outcome gates re-enabling those four
deferred optimizations and Option A.
Tested grid: hidden={512,768,1024,1536} × layers={3,4,6,8} subset.
200 iterations per config on home (RTX 3090), single seed initially,
second seed for boundary configs. Eval cadence held at default.
Decision tree included for: success → recommend new default and
trigger downstream plans; null → document and stay; expensive-but-
better → defer until AMP/compile/TRT land.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
performance.md additions:
- Batched Traversal Inference design decision (A vs B vs C with
rationale).
- Option A bench result and structural ceiling (realized batch ~7.2,
IPC overhead exceeds GPU gain at small model size).
- Post-A optimization calculus: why compile/TensorRT remain
iter-neutral today and become meaningful only after model growth
and/or denser eval. Sequencing matters; do not retest these on the
current small model.
- Free-threaded Python (3.13t/3.14t) note: cleanest endpoint in
principle, but PyTorch maturity + Cython nogil audit cost block
near-term adoption.
docs/plans/ (4 plan documents for Codex execution):
- batched_traversal_inference_server.md (executed; deferred).
- amp_trainer.md.
- torch_compile.md.
- cython_safe_heuristic_bots.md (executed; first-pass landed).
docs/reports/ (3 cost reports):
- cost_pytorch_free_threaded_2026-05-07.md: WAIT 3-6 months;
PyTorch wheels exist but our Cython is the gating cost.
- cost_cython_nogil_audit_2026-05-07.md: medium effort, traversal.pyx
carries 90% of blockers; Steps 1-3 (cfr_math/encoding nogil
keywords, TraversalStats cdef class) are safe and cheap, Steps
4-6 wait for triggers.
- cost_pytorch_cuda_multithread_2026-05-07.md: risky;
optimizer.step / load_state_dict race silently with concurrent
forward; per-thread default streams unset means naive threading
serializes on default stream anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Measured DeepCFRMLP forward at bs={1,4,16,64,256,1024} on RTX 3090.
Per-state cost drops 232× from bs=1 (80 µs) to bs=256 (0.34 µs) while
per-call latency stays near 90 µs through bs=256. Policy-call supply
from a real run is ~368 states per traversal and ~200k per iteration,
well above the bs=64–256 plateau, so batched inference is not
supply-limited. GPU forward is not the limiter once batching exists.
Verdict: Optimization Priorities #5 (batched traversal inference) is
worth pursuing. End-to-end gain will still be bounded by encoding and
worker-GPU coordination overhead.
- scripts/profile_gpu_forward.py: standalone profiling script
- docs/performance.md: new "GPU forward profiling for batched traversal"
experiment section with table, supply estimate, and verdict
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Make the link from the failed torch.compile experiment to Optimization
Priorities #5 explicit, so a future revisit happens at the right time
(once compile is on the dominant phase, not just trainer optimization
steps).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wrapping trainer networks with torch.compile produced a 4.8% regression
in iteration time on default.yaml (17.93s → 18.79s). Two causes: (1)
the dominant phase is CPU traversal which bypasses the compiled
wrapper, (2) DeepCFRMLP is too small for compile dispatch overhead to
pay back. Implementation kept on experiments/torch-compile for future
revisits when the trainer model or inference path changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
같은 hyperparameter (512x3, 2x updates, argmax_tiebreak)에 opponent_policy
만 self_play_league로 다른 run과의 iteration별 비교 표 추가. iter 350
시점에서 Random WR 38%p 격차 (network 34% vs league 72%) 확인 — 발산
가설을 실증.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
opponent_policy=network 설정으로 1000-iter 실험 두 개 (512x3, 1024x4)를
돌린 결과 두 실험 모두 policy collapse가 발생함을 확인. 큰 capacity는
plateau를 늘리지만 발산 자체를 막지 못함. 향후 학습은 self_play_league
기본값을 유지할 것을 권고.
- 실험에 사용한 config 두 개 추가
- 발견 분석 문서 추가 (원인, 비교, 권고)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
coolguyandClaude Opus 4.7