The earlier exploiters ran a quarter of their targets' budget. Rerun at a matched
~131M learner actions (measured: 130.4M and 131.1M):
ours exploiter reaches 0.4657 [0.455, 0.477], mean lead -5.5
league exploiter reaches 0.6295 [0.619, 0.640], mean lead +28.8
The CIs are nowhere near each other and the gap widened under funding, 0.094 ->
0.164. A qualitative line gets crossed too: a properly funded attacker trained
from scratch *beats* league outright, and still loses to ours.
Two things a reader has to carry with that, both of which cut against us:
- Neither exploiter plateaued -- both climb through the final update, so these
remain lower bounds. And ours' attacker has the steeper tail (+0.031 vs +0.016
per 100 updates) while league's curve is already bending, so a much larger
attacker could narrow the gap. Cannot rule out that it closes.
- League is a single-round policy blind to carry and match score, and both frozen
policies are played greedily, which is maximally exploitable by construction.
So this is not a clean "training method A vs B".
The claim that survives is the matched-budget one, and only that one.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
The exploiters ran 32.5M learner actions, not the 65.5M I recorded -- they train
one seat, so a 250x1024 run yields half of what the same shape gives the
both-seat self-play trainer. Against targets trained on 131M and 122.6M, that
makes the attacker roughly 4x underfunded.
Which means the absolute number does not support "ours is only 22.8%
exploitable". It supports exactly one claim: at a matched budget, league gives up
more. Whether the ordering survives a properly funded attacker is now the open
question, so the script takes --updates and --batch-games to run it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
Winning the head-to-head says a policy is strong on average, not that it is hard
to beat. So freeze each policy, train a fresh one from scratch whose only job is
to beat that policy, and see how far it gets.
Same exploiter budget (250 updates x batch 1024, 65.5M learner actions):
ours (match stack, 131M) exploiter reaches 0.2278 [0.219, 0.237]
league (web-deployed) exploiter reaches 0.3213 [0.311, 0.331]
League gives up 9.4 more points to a dedicated attacker. Both sit far from 0.5, so
neither is a pushover -- but the caveat left open by the head-to-head is now closed
in our favour on both axes: stronger on average (0.6094) and harder to exploit.
Worth noting against expectation: league was trained *with* an exploiter structure
and we ran pure self-play, and we still come out less exploitable. Whatever the
league machinery buys, it did not buy that here.
The number is a lower bound -- a bigger exploiter would find more -- so it only
means anything as a like-for-like comparison, which is how it is used.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
The 39.3M ablation was the critic's worst ground. At a matched 131M budget:
both-seat training is worth +17.1 points of win rate, the privileged critic
+3.7 -- a 4.6x gap. And they behave differently: both-seat is flat across scale
(+16.8 at 39.3M, +17.1 at 131M) while the critic swings (-1.6 to +3.7).
The mechanism explains it. Both-seat doubles the data at the same compute --
self-play already played those plies with the same network. The critic adds no
data, only accuracy, and a privileged value trunk has more to fit, so it grades
badly until there is enough to fit it with.
Fable ranked the critic first among missing ideas and both-seat fourth. That was
exactly backwards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
The 39.3M ablation said the privileged critic hurt (switching it off won 0.5160
[0.505, 0.527]). Head to head at 131M, both sides trained identically, it says the
opposite: off *loses*, 0.4633 [0.453, 0.474]. Against league the critic-on model
scores 0.6094 and the critic-off one 0.5526.
The critic earns its keep once there is enough data to fit it -- at 39.3M the
privileged value trunk is underfit and only adds advantage noise. Defaulted back
on, with the small-scale number kept in the docstring as a warning: an ablation at
a budget you do not intend to ship can invert.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
Each piece switched off in turn, trained at identical compute, then played against
the full stack over 8192 duplicate matches. Below 0.5 means the removed piece was
doing work.
- both seats: 0.3317 [0.322, 0.342]. The biggest single contributor. Half of it is
simply sample count -- dropping the opponent seat halves the learner actions per
update -- but that is the point: self-play already produced those plies with the
same network, and the old trainer stop_gradiented them away.
- match observation: 0.4751 [0.464, 0.486]. Small but real. Since carry itself
contributes almost nothing (rounds decompose), most of this is likely the
single-round observation defects being fixed: to_move, the deck clock, and the
score_diff scale.
- privileged critic: 0.5160 [0.505, 0.527] -- turning it OFF makes the agent
significantly STRONGER. Fable called this the biggest missing idea; it is wrong.
A critic that knows the deck fits V(full state), which is not
E[return | masked obs], so the advantage picks up a component the actor cannot
act on. From the actor's side that is noise, not variance reduction. Asymmetric
critics hurting under partial observability is a known failure mode.
Defaulted off accordingly. (Reusing it as a PIMC leaf evaluator may still stand --
that is a separate claim from using it to train the policy.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
The question was what any of this actually improved over the training method
that existed. Duplicate matches, 8192 a piece, same three deals and coins from
both seats:
- At 39.3M learner actions the match stack beats the Phase 0a gate agent
(0.5842) which had 411M -- 10.5x the data. Sample efficiency is the headline.
- At 39.3M it *loses* to the league policy (0.3142). That is a budget gap, not a
strength gap: league had 122.6M plus a league/exploiter structure.
- Scaled to a matched budget (131M vs league's 122.6M) it wins: 0.6094
(CI 0.599-0.620), +22.0 points.
So: same compute, stronger agent, measured on the actual game.
Caveat kept honest in the plan -- league was trained with exploiters, and we have
measured average strength, not exploitability. "Harder to exploit" is not shown.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
The user asked why we were not simply maximising the three-round total, and they
were right. Each ply now pays the points by which it moved the running match
difference; at gamma=1 that telescopes to the final total, so the objective is
exactly the rulebook's, handed out every ply instead of once 150 plies later.
Three measurements killed the tanh design:
- Rounds are independent (corr(m1,m2)=0.004, corr(m1+m2,m3)=0.05), so a reward
linear in the total decomposes the match into three independent rounds and
carry enters the objective nowhere. The only coupling, the start-player rule,
is worth +0.73 +/- 0.84 points -- indistinguishable from zero.
- Risk attitude, the one thing tanh buys, is worthless here. A policy made to
gamble when it trails by 20 entering round three *loses* to a greedy clone over
6144 duplicate matches (0.482); gambling only at -40 breaks even (0.498). A
marginal wager buys about +1.7 sigma for -2 to -3 expected points. Ceiling on
the whole carry-conditioning idea: under one win-rate point.
- Head to head over 10,000 duplicate matches at equal compute, the linear reward
*beats* tanh(total/12): 0.5859 (CI 0.576-0.596), +20.3 points. Dropping it is
not merely free, it is better -- not because of risk, but because tanh hands a
~150-ply match one saturated +/-1 and leaves all credit assignment to the critic.
The flat carry probe was not exploration collapse: sampled play still opens 5.00
expeditions, entropy settles at 1.36 nats (3.9 effective actions), and the critic
reads carry cleanly (round-three values run -0.87 to +0.86, monotone). The signal
was there; there was nothing to buy with it.
Criterion 1 (a monotone carry response) comes off the gate accordingly -- the
optimal response barely exists in this game. carry stays in the observation: it
costs nothing and the start-player rule keys off it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
Self-play works: matches converge to 146.8 plies (~49 a round) with a 91% deck-
race rate, so the stalling that static opponents induced is gone. Duplicate match
eval scores 0.4968 with a mean lead of exactly 0.0 -- same deals, same coins, both
seats, deal luck cancelling exactly.
Success criterion 1 does not pass. The carry probe is close to flat: expeditions
opened sit at 5.00 whether the policy is 60 points down or 60 points up. Wager use
does move monotonically across all six carry levels, and in the right direction
(behind -> more multipliers), but the spread is 0.31 wagers.
Two diagnoses, one of which was mine and wrong:
- Residual potential shaping was NOT the cause. Annealing it fully to zero left
the probe just as flat.
- terminal_scale is. At carry -60, tanh((margin - 60)/50) is close to linear over
any realistic round margin, and maximising E[tanh] on a linear stretch is just
maximising E[margin] -- there is no reason to gamble. Risk-seeking only appears
where tanh is sharply convex, which needs a smaller scale. Dropping 50 -> 12
widens the wager spread 0.19 -> 0.31, which is the mechanism showing up.
The probe itself is also mis-scaled: at scale 12, tanh(60/12) is 1.0, so +/-60 is
a saturated dead zone with no gradient and the policy has learned nothing there.
The measurable band is |carry| <~ 2 * terminal_scale, and the probe levels have to
be set from the scale rather than fixed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
The cap item existed so a three-round match would fit inside a 400-step scan.
Phase 0a's truncation bootstrap removed that constraint -- scan length is a free
parameter now -- and the same phase showed the cap is the thing that turns
stalled games into losses: 20.6% of cap-hit games lost, 99.4% of all losses. A
120-ply cap would put *more* games into the wall and make "freeze the round
while ahead" easier, so the round cap stays at 400 and max_steps_rate stays a
watched metric.
Also records what the plan never listed: ppo.py, gates.py and league.py all take
a single-round State and still have to be wired to MatchState. That is the
largest remaining piece of Phase 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
At equal compute (250 updates either way, 5.8x the samples) the new loop beats
the baseline against heuristic_expert: 0.4525 -> 0.5071 win rate, -4.28 ->
-0.56 mean score diff. Expert ends rounds at their natural length, so that is
a clean read on card play.
It appears to *lose* ground against heuristic_balanced (0.9866 -> 0.9292), but
that is entirely the MAX_STEPS=400 cap. Drawing from a discard pile does not
deplete the deck, so a round can be stalled indefinitely; against a weak
opponent the extra turns are worth points, and with 5.8x the samples the agent
learns the exploit harder (greedy rounds run 225 plies). Splitting 2048 games:
naturally-ended games lose 0.08%, cap-hit games lose 20.62%, and 99.4% of all
losses land in cap-hit games. The board freezes mid-expedition and the -20s
stand.
Self-play does not have this failure mode -- league runs converge to 53.6-ply
rounds and never hit the cap -- which is the regime the 3-round work targets.
Noted in the plan: static heuristic anchors are unusable as gates, and the
per-round cap needs care in Phase 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
rollout_steps was pinned to MAX_STEPS (400) while a round actually runs
~50 plies, and finished envs were only reset between updates. 83% of every
rollout was spent stepping already-done envs to produce masked-out zeros.
Measured at rollout_steps=400: active steps go 17.3% -> 100%, i.e. 5.8x the
learner actions per update for the same compute.
Resetting in-scan exposes three things that were previously benign:
- compute_gae bootstrapped truncated episodes from zero. That was safe only
because every episode used to terminate inside the scan; now episodes cross
the boundary, so thread V(s_T) through.
- rollout_metrics read final_env and summed rewards along the scan axis, both
of which assume one episode per slot. With several episodes per slot that
silently produces garbage, so aggregate at done boundaries instead.
- league assignments were redrawn only between updates, which would pin a slot
to one seat/opponent across every episode in a scan. Redraw them on reset.
Also anneal potential shaping against learner actions rather than padded scan
steps: the old accounting counted the dead steps, so a 5M-step anneal expired
within two updates of 250. Any earlier evidence that shaping does not help was
gathered with it effectively off.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
Mini Lost Cities (3색 5랭크) 50 iter / 20 eval games:
- score_diff vs random +33, vs heuristic_cautious -5.7 (45% 승률)
- play_action_rate 13~31% (Deep CFR의 0~2% 대비 명확)
- trap escaped on mini, "long-horizon credit assignment" 가설 지지
Codex commit e69f316으로 ISMCTS 구현 완료. Full game scale-up이
다음 후보 (100 iter ETA 3시간 추정).
R2 167 iter 시점 trap signature 명확히 R0/R1과 동질 (play_action_rate
≈ 0%, positive expedition 0.02/game, bad_open 0.98). 세 가지 opponent
환경(self / passive / competent) 모두에서 같은 결함 확정. opponent
dimension closed.
Doc §11에 R2 결과 + 세 런 비교표 + opponent 변경으로 풀 수 없음
결론 + 다음 방향(SO-ISMCTS 정공법) 기록.
이전 레포(../coolrl) commit 5a98855, da0d4cd에서 시도된
regret_matching_epsilon 튜닝(eps1e3 → eps1e4)이 zero-pit 대응책이었고
1e-4가 best로 채택되어 현재 default 0.0001로 남아있음을 doc에 기록.
이 lever는 이미 당겨진 상태이며 R1의 새 zero-pit은 다른 원인
(opponent=discard_only가 만든 game-theoretic 균형)이라는 점 명시.
R1 결론: trap 깨지지 않음. 자기참조 회로는 충분조건이 아니라 trap 강화
요인일 뿐. opponent를 fixed external로 바꾸자 모델이 zero-pit으로
collapse — 핵심 결함은 model 자체의 credit assignment 실패임이 확정.
R2 방향 = curriculum (작은 게임 → 큰 게임)을 후속 후보로 등록.
Three coordinated hygiene changes; none target the diagnosed
selection-bias bottleneck. They make the codebase honestly reflect the
pure-self-play stance and reduce dashboard noise.
Bot rename (drop the unhelpful safe_ prefix; suffixes describe behaviour):
- safe_heuristic_loose -> heuristic_aggressive
- safe_heuristic -> heuristic_balanced
- safe_heuristic_strict -> heuristic_cautious
- noisy_safe -> heuristic_noisy
- passive_discard -> discard_only
Class renames in bots/: SafeHeuristicBot -> HeuristicBot,
SafeHeuristicParams -> HeuristicParams, PassiveDiscardBot -> DiscardOnlyBot,
plus loose/strict parameter constants. Backwards compatibility was dropped
intentionally per user instruction; no aliases. Active configs, docs,
scripts, tests updated. Archive directories (configs/archive,
docs/archive, runs/archive) left intact and may still reference old
names per their read-only policy. The src/.../bots/passive.py module was
renamed to discard_only.py via git mv.
Analyze plot curation (deep_cfr/analyze.py):
- Added analysis_00_core.png as the canonical daily dashboard with 10
heuristic-free metrics (loss/{advantage,strategy}; vs heuristic_cautious:
avg_score_diff0, win_rate0, avg_opened_colors, positive_expedition_rate,
bonus_expedition_rate, score_per_opened_color, policy_entropy; vs random:
win_rate0).
- Removed analysis_05_open_quality.png (bad/weak/good open rates,
recoverable score) and analysis_07_calibration.png (calibration gap,
recoverable mean) - both relied on the heuristic recoverable_score
classifier already dropped from inputs.
- Removed SELECTIVITY_PLOTS and plot_selectivity (heuristic-laden).
- SUMMARY_EVAL_METRICS no longer includes bad_open_rate or
calibration_gap.
- PlotSpec gained an opponents allowlist so the new core section can pin
a specific opponent per panel without restructuring plot_section.
Tiered evaluation cadence (EvaluationConfig):
- Added extended_opponents and extended_eval_every (default 0 = disabled).
- opponents_for_iteration(iteration) returns the core list every
eval_every and appends extended_opponents (de-duplicated) when
iteration is also a multiple of extended_eval_every.
- default.yaml now uses 3 core opponents (random, discard_only,
heuristic_cautious) every 5 iterations and 3 extended opponents
(heuristic_balanced, heuristic_aggressive, heuristic_noisy) every 50
iterations. random is the floor sanity. discard_only is the
zero-pit detector / absolute-score reference (its score is always 0,
so eval/discard_only/avg_score_diff0 directly equals the model's raw
average score). heuristic_cautious is the ceiling and the
archive-comparable benchmark used in the prior diagnostic sections.
Net eval cost reduction: roughly 50% (3 opponents x every 5 iter, plus
6 opponents x every 50 iter, vs the prior 6 x every 5).
Documented in docs/plans/deep-cfr-selectivity.md section 9.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The interleaved traversal scheduler's _regret_matching was hard-coded to
spread fallback policy uniformly across legal actions, regardless of the
configured regret_matching.all_negative_fallback. default.yaml has
shipped with all_negative_fallback: argmax_tiebreak since 618d5f8 based
on the 20-iter audit + 1000-iter empirical comparison in
docs/archive/deep-cfr-regret-fallback-audit-2026-05-07.md, but the
default scheduler was switched to interleaved in 09bbe7c, after which
the configured fallback mode silently no-op'd.
_regret_matching now takes fallback_mode and concentrates policy mass on
the lowest-index tied action when "argmax_tiebreak". Tiebreak is
deterministic; the Cython recursive traverser randomises ties using its
per-traverser RNG, which the batched policy does not have. Behaviour
matches the spirit of the recursive path (concentrate on best, do not
dilute uniformly).
Plumbed through BatchedPolicy, InterleavedTraversalConfig,
run_interleaved_traversal_batch, trainer.py, workers.py, and the
analyze_first_open_targets.py caller. Two unit tests added.
Also bumps default.yaml outcome_sampling_epsilon 0.2 -> 0.05. The
200-iter sweep in docs/plans/deep-cfr-selectivity.md section 1 showed
0.05 produced the best short-run safe_heuristic_strict score diff
(-40.01 vs -57.87 for 0.20). Recent experiments already used 0.05; the
default now matches actual experimental practice.
Neither change targets the diagnosed selection-bias bottleneck. They
align config intent with scheduler behaviour and make the default config
reproduce known-best knob settings out of the box.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ColorSharedNetwork previously sliced the input vector into n_colors equal
chunks (input_dim // n_colors). The slice boundaries do not align with the
actual encoding layout: adjacent slices contain phase flags, hand slots,
expedition state, scores, etc. mixed together. The "color-shared" encoder
was therefore sharing weights across semantically unrelated chunks, not
across per-color blocks. The single archived run that exercised this path
(2026-05-07_092137_color_shared_attention_1000iter) was killed at iter 41
and produced no eval data, so we have no measurement of whether a real
per-color architecture would help.
Adds compute_lost_cities_color_layout(input_dim) which returns explicit
per-color and common index lists for the standard Lost Cities encoding
(n_colors=5, hand_size=8, n_ranks=9). It recognises input_dim values
171, 219, 249, 297 across derived_playability and slot_aware_playability
flag combinations.
Per-color block (39 dims when derived_playability is on): both players'
expedition state for that color, discard top, public-histogram row,
pending-discard one-hot bit, legal-action draw-pile bit, and the
derived_playability per-color block. Slot-aware features are slot-major
and stay in common.
ColorSharedNetwork.forward now indexes per-color blocks via the layout
when input_dim matches a known schema. For other dims (unit tests,
non-Lost Cities use), it falls back to chunked slicing with a UserWarning
- preserves backward compatibility for tests but makes the legacy
behaviour visible.
No fair test of the new architecture was run as part of this commit.
Documented in docs/plans/deep-cfr-selectivity.md section 7.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Audit of the Deep CFR information-state encoding identified two tiers of
non-pure features and removed both:
- Tier 3 (judgment): is_bad_open_candidate, open_risk_score,
is_safe_continuation. Same heuristic family used to label bad_open in
evaluation, embedded as model input.
- Tier 2 (projection): recoverable_score_no_bonus,
recoverable_margin_no_bonus, min_needed_to_break_even,
cards_needed_for_bonus, has_bonus_path. Mechanical but assumption-laden
("commit and play all currently-playable cards"). The no_bonus form is
asymmetric: it amplifies the immediate -20 penalty while truncating the
+20 bonus upside, biasing the model toward the same "don't open" basin
the diagnostics already flagged.
Input dim 365 -> 297. DERIVED_PLAYABILITY_PER_COLOR 19 -> 15;
SLOT_AWARE_PLAYABILITY_PER_SLOT 12 -> 6. Test shape assertions updated.
Also adds selectivity diagnostic infrastructure used to reach this point:
- traversal.outcome_unsampled_first_open_prior_alpha config field with
signed-prior overlay on unsampled first-open advantage targets (A1).
- analyze_first_open_counterfactual.py --post-policy to swap the
policy_player rollout policy and isolate selection bias (D1).
- analyze_first_open_followup.py to inspect post-forced-open behavior
(E2): same-color play vs discard counts, other-open rate, terminal
hand composition.
Findings recorded in docs/plans/deep-cfr-selectivity.md sections 3-6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second survey processed the remaining 12 archives via gemini after
the first batch of 3 was accepted. 12 drafts, 0 skips, 0 errors.
Every draft carries a deterministic Last-verified header
(2026-05-08, commit 5c221fb) thanks to the post-processing fix
landed in the previous commit. All 12 accepted into docs/research/
verbatim:
deep-cfr-evaluation-profile-plan
deep-cfr-legacy-experiment-reproduction
deep-cfr-legacy-runtime-comparison
deep-cfr-performance-experiments
deep-cfr-profile-advantage-memory-split
deep-cfr-profile
deep-cfr-regret-fallback-audit
deep-cfr-v0-gap-vs-coolrl
deep-cfr-v0-plan
fast-engine-next-optimizations
post-a-optimization-calculus
test-coverage-notes
docs/archive/ is now fully covered: every entry either has a
research counterpart by stem or by tail-match.
Also extracts _dispatch_one and adds --parallel N to
scripts/librarian_survey.py. ThreadPoolExecutor over the per-archive
work is safe because subprocess.run is network-bound (no GIL fight)
and each thread writes to its own output filename. Default stays
1 (sequential); --parallel 4 is the recommended speedup for large
surveys. The two surveys above ran sequentially; future runs can
opt in.
Plan declares librarian closed for new feature work. MEMORY drift
fixup and duplicate-merge modes stay deferred until a real input
surfaces. Stage 1 (5 deterministic checks) and Stage 2 (promote +
survey) remain operational.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Spot-check of the three drafts gemini produced in the --max 3
survey smoke test: all cited file paths exist, line numbers and
function names land within 1-2 lines of actual symbols
(game.pyx:217 cdef class GameState, evaluate.py:220 batched-entropy
block, trainer.py:892 _evaluate_parallel, action_distribution at
evaluate.py:238 with cited code at line 250 inside it). Numbers
cross-checked against archives match. Conclusions preserved.
The one systemic weakness was the Last-verified commit field:
gemini left a `<short-hash>` placeholder, a literal `HEAD`, or
omitted the commit entirely depending on the call. Fixed in two
places:
1. Manually patched the three drafts before acceptance and copied
them into docs/research/.
2. Added _current_commit_sha and _post_process_draft helpers to
both librarian_survey.py and librarian_promote.py. The drafts
now go through `**Last verified:**` line normalization that
substitutes today's date and `git rev-parse --short HEAD`
before being written to disk. Future runs converge
deterministically.
Net: docs/research/ gains classic-port-notes.md,
deep-cfr-batched-evaluation.md, and deep-cfr-evaluation-profile.md.
12 archive entries remain unprocessed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/librarian_survey.py walks docs/archive/*.md and dispatches
every entry without a research counterpart through the same prompt
assembly as librarian_promote.py. Outputs land under
runs/tmp/librarian-survey-<timestamp>/, classified into:
<stem>.md draft, ready to copy into docs/research/
<stem>.SKIP.txt LLM's one-line "not promotable" reason
<stem>.ERROR.txt CLI stderr if the call itself failed
Counterpart detection uses exact stem match plus a tail-match
heuristic so research notes that intentionally drop a domain prefix
still suppress their archive. Verified against the current tree:
docs/archive/deep-cfr-opponent-policy-network-divergence-* is
correctly recognized as already covered by
docs/research/opponent-policy-network-divergence.md.
--dry-run lists candidates and suggested research targets without
calling the LLM. --max N caps processed archives per run, useful as
a cost guard. Sequential dispatch; one LLM call per archive.
Plan updated to mark survey mode complete.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/librarian_promote.py gains a --accept flag that copies the
generated draft to the suggested docs/research/ target in the same
invocation. This is explicit per-invocation opt-in, not auto-apply:
the operator types --accept knowingly, the cp is still a deliberate
acceptance decision, just expressed in one command instead of two.
docs/research/option-a-bench-result.md is the first promoted note,
generated by gemini from
docs/archive/option-a-bench-result-2026-05-07.md and accepted
verbatim. Spot-check verified that cited file:line locations match
current source (traversal.pyx:473-475 and
inference_server.py:226-228 carry the cited code), the Last-verified
header reflects today's date and HEAD, and the durable conclusion
(sync-blocking policy boundary as the structural ceiling, not IPC
plumbing) is preserved.
This closes the end-to-end loop the librarian was designed for:
oversize check surfaced docs/performance.md, the routing pass split
durable analysis into the archive entry, the promote dispatcher
turned the archive entry into a research draft, and the human
accept step landed it as a tracked research note. Took one LLM call.
Removes the now-resolved ignore-list entry for
docs/research/option-a-bench-result.md (the file exists; the
forward reference is real).
scripts/librarian.sh exits 0 against the working tree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/librarian_promote.py is the first Stage 2 piece: a
vendor-agnostic LLM dispatcher that drafts a docs/research/ note
from a given docs/archive/ entry. It assembles the prompt by
stitching scripts/librarian-prompt.md (system) onto the archive
body with a "draft a research note per the rules above" task
instruction, then shells out to the CLI selected by LIBRARIAN_LLM
({claude|codex|gemini}; default claude). The LLM's stdout is
captured to runs/tmp/librarian-promote-<timestamp>-draft.md for
human review — the script never writes into docs/research/ itself.
Refusal cases:
- path not under docs/archive/
- target docs/research/<stem>.md already exists (after stripping any
-YYYY-MM-DD suffix)
- archive missing
If the LLM judges the archive non-promotable, it is instructed to
return a single SKIP: <reason> line instead of fabricating a draft.
--show-prompt prints the assembled prompt without invoking the LLM,
useful for inspecting what would be sent.
Plan updated: Stage 2 v1 marked complete; remaining Stage 2 work
(MEMORY drift fixup, duplicate-merge, survey mode) catalogued.
Next concrete step is a smoke test against one real archive entry.
Adds one ignore-list entry for docs/research/option-a-bench-result.md
which appears in the plan as a hypothetical accept target.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
docs/performance.md grew to 914 lines because dated experiments and
design analyses kept getting appended instead of routed to
docs/archive/ and docs/research/ as AGENTS.md prescribes. The
oversize check from librarian Stage 1 surfaced the file; this commit
acts on that finding by extracting the parts that belong elsewhere
and trimming the source to a focused current-state reference.
Extracts (verbatim from the original prose, with cross-link headers
and a brief routing note added at top):
- docs/archive/deep-cfr-performance-experiments-2026-05-07.md
bundles torch.compile (regression), AMP (regression), GPU forward
profiling (decision support), and Option B interleaved traversal
(pass) — same date, same theme.
- docs/research/batched-traversal-inference-decision.md captures the
durable A vs B vs C rationale with a closing "Outcome" pointer to
the post-bench archive doc.
- docs/archive/post-a-optimization-calculus-2026-05-07.md preserves
the forward-looking sequencing recorded pre-bench.
- docs/archive/option-a-bench-result-2026-05-07.md preserves the
regression diagnosis and re-enable criteria.
docs/performance.md is now 345 lines, holds sections 1–9 (current
runtime / bottleneck / device / AMP status / batching / eval /
TensorRT / priorities), and ends with a "See Also" linking the four
extracts.
Also reworded the AGENTS.md soft-cap rule from a bare "~500-line
soft cap" to clarify the intent: the cap is a *routing trigger* (is
content piling up that should live in archive/research?), not a
split mandate. Reduces the risk of future agents shredding a useful
doc just to satisfy a number.
scripts/librarian.sh now exits 0 against the working tree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the last two deterministic Stage 1 checks and wires them into
scripts/librarian.sh:
- librarian_check_stale_plans.py: per docs/plans/*.md, uses
`git log -1 --format=%cs` to get the last commit date and flags
plans untouched in 60+ days. Plans should be either active or
archived; long silence is drift.
- librarian_check_memory_drift.py: validates the user-memory dir
(~/.claude/projects/<slug>/memory). Each MEMORY.md index line
must point at a real file with frontmatter (name, description,
type ∈ {user, feedback, project, reference}); no orphaned memory
files. Memory dir is derived from repo root for portability.
Both run clean against current state.
Stage 1 declared complete. Promotable-archive and duplicate-prose
detection are moved to Stage 2 in the plan because both need LLM
judgment to avoid false positives — not pattern matching.
Next concrete step is the open Stage 1 finding from yesterday:
docs/performance.md at 914 lines, due for a split into sub-topic
notes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Third Stage 1 piece: scripts/librarian_check_oversize.py walks
non-archive markdown files and flags any over the 500-line soft cap
declared in AGENTS.md. Wired into scripts/librarian.sh.
Caught one real finding on first run: docs/performance.md at 914
lines. Splitting it into sub-topic notes is a separate cleanup task
— surfaced for the user, not auto-applied.
Updates docs/plans/librarian.md Progress + sets the next concrete
step to a stale-plan checker.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single entry point that runs every Stage 1 check in order and
aggregates exit codes — both checks always run so users see all
findings in one pass. Two checks wired up today (lychee link
integrity, file:line citations), more land incrementally as Stage 1
grows.
Removes scripts/librarian.sh from the ignore list now that the file
exists. Updates docs/plans/librarian.md Progress + Next Step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/librarian_check_citations.py extracts file:line references from
inline-code spans across docs/**/*.md and verifies each path exists
(and, if cited with a line number, is within range). Skips
docs/archive/ and docs/plans/archive/ which are read-only by policy.
Caught one real drift in docs/research/optimization_sequencing.md: the
note pointed at docs/plans/amp_trainer.md, which had moved into
docs/plans/archive/.
scripts/librarian-ignore.txt holds fnmatch globs for citations that are
intentionally future-tense (planned files described in the plan docs
themselves). Used sparingly so the checker stays useful as a drift
signal.
Updates docs/plans/librarian.md Progress + Next Step. Next concrete
step is a thin scripts/librarian.sh orchestrator over both checkers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the full librarian design (two-layer architecture, three-stage
pipeline, vendor-agnostic via LIBRARIAN_LLM env, propose-only / no
auto-apply) in docs/plans/librarian.md. Lands the first concrete Stage 1
piece: scripts/librarian_check_links.py, a lychee --offline wrapper
ported from ~/dev/coolrl/src/coolrl/dev/check_doc_links.py.
Also moves the librarian prompt from .claude/agents/ (Claude Code only)
to scripts/librarian-prompt.md so any CLI can load it as a system
prompt later. Fixes one stale README link the new checker caught:
docs/classic-port-notes.md → docs/archive/classic-port-notes.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an experiment-only explicit-continuation traversal prototype that batches policy requests, records CPU/CUDA parity results, and updates the Option B plan with Phase 0/1 evidence.
Co-Authored-By: Codex <codex@openai.com>
Archive implemented AMP, Option A inference-server, and Cython heuristic plans. Add the active Option B interleaved traversal plan and update model-size/torch.compile plans to reflect the current traversal scheduling conclusion.
Co-Authored-By: Codex <codex@openai.com>
The experiment finds a model size where (a) learning-curve gains
justify compute, and (b) forward time is large enough to amortize
AMP/compile/TensorRT overhead. Outcome gates re-enabling those four
deferred optimizations and Option A.
Tested grid: hidden={512,768,1024,1536} × layers={3,4,6,8} subset.
200 iterations per config on home (RTX 3090), single seed initially,
second seed for boundary configs. Eval cadence held at default.
Decision tree included for: success → recommend new default and
trigger downstream plans; null → document and stay; expensive-but-
better → defer until AMP/compile/TRT land.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
performance.md additions:
- Batched Traversal Inference design decision (A vs B vs C with
rationale).
- Option A bench result and structural ceiling (realized batch ~7.2,
IPC overhead exceeds GPU gain at small model size).
- Post-A optimization calculus: why compile/TensorRT remain
iter-neutral today and become meaningful only after model growth
and/or denser eval. Sequencing matters; do not retest these on the
current small model.
- Free-threaded Python (3.13t/3.14t) note: cleanest endpoint in
principle, but PyTorch maturity + Cython nogil audit cost block
near-term adoption.
docs/plans/ (4 plan documents for Codex execution):
- batched_traversal_inference_server.md (executed; deferred).
- amp_trainer.md.
- torch_compile.md.
- cython_safe_heuristic_bots.md (executed; first-pass landed).
docs/reports/ (3 cost reports):
- cost_pytorch_free_threaded_2026-05-07.md: WAIT 3-6 months;
PyTorch wheels exist but our Cython is the gating cost.
- cost_cython_nogil_audit_2026-05-07.md: medium effort, traversal.pyx
carries 90% of blockers; Steps 1-3 (cfr_math/encoding nogil
keywords, TraversalStats cdef class) are safe and cheap, Steps
4-6 wait for triggers.
- cost_pytorch_cuda_multithread_2026-05-07.md: risky;
optimizer.step / load_state_dict race silently with concurrent
forward; per-thread default streams unset means naive threading
serializes on default stream anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
coolguyandClaude Opus 4.8