Accept first survey batch + add commit-hash post-processing

Spot-check of the three drafts gemini produced in the --max 3
survey smoke test: all cited file paths exist, line numbers and
function names land within 1-2 lines of actual symbols
(game.pyx:217 cdef class GameState, evaluate.py:220 batched-entropy
block, trainer.py:892 _evaluate_parallel, action_distribution at
evaluate.py:238 with cited code at line 250 inside it). Numbers
cross-checked against archives match. Conclusions preserved.

The one systemic weakness was the Last-verified commit field:
gemini left a `<short-hash>` placeholder, a literal `HEAD`, or
omitted the commit entirely depending on the call. Fixed in two
places:

1. Manually patched the three drafts before acceptance and copied
   them into docs/research/.
2. Added _current_commit_sha and _post_process_draft helpers to
   both librarian_survey.py and librarian_promote.py. The drafts
   now go through `**Last verified:**` line normalization that
   substitutes today's date and `git rev-parse --short HEAD`
   before being written to disk. Future runs converge
   deterministically.

Net: docs/research/ gains classic-port-notes.md,
deep-cfr-batched-evaluation.md, and deep-cfr-evaluation-profile.md.
12 archive entries remain unprocessed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-08 01:52:08 +09:00
co-authored by Claude Opus 4.7
parent b0b385591b
commit 5c221fb3c6
6 changed files with 209 additions and 5 deletions
+27
View File
@@ -235,6 +235,33 @@ non-promotable), or `<stem>.ERROR.txt` (CLI failure).
caps the number of archives processed per run, useful for smoke
tests or cost control.
### Smoke test results (2026-05-08, gemini, --max 3)
3 drafts, 0 skips, 0 errors. Spot-check verified:
- All cited file paths exist; line numbers and function names
resolve to within 12 lines of the actual symbols
(`game.pyx:217` `cdef class GameState`, `evaluate.py:220`
batched-entropy block, `trainer.py:892` `_evaluate_parallel`,
etc.).
- Numbers cross-checked against archives match (61.83/38.92/14.83
eval seconds; 4.05/2.88 advantage train seconds).
- Conclusions preserved.
- One systemic weakness: gemini did not run `git` to resolve HEAD,
leaving `commit <short-hash>` placeholder, literal `commit
\`HEAD\``, or omitting the commit field. Fixed in two places:
the three drafts were patched manually before acceptance, and
both `librarian_promote.py` and `librarian_survey.py` now
post-process the LLM's output to rewrite the `**Last verified:**`
line with the real short SHA from `git rev-parse --short HEAD`
before writing to disk. Future runs converge to the right header
deterministically.
All three drafts accepted into `docs/research/`:
`classic-port-notes.md`, `deep-cfr-batched-evaluation.md`,
`deep-cfr-evaluation-profile.md`. 12 archive entries remain
unprocessed for the next survey run.
## Stage 2 remaining
- MEMORY.md drift fixup mode (read drift report, propose one-line