Complete librarian Stage 1: stale plans + MEMORY drift

Adds the last two deterministic Stage 1 checks and wires them into
scripts/librarian.sh:

- librarian_check_stale_plans.py: per docs/plans/*.md, uses
  `git log -1 --format=%cs` to get the last commit date and flags
  plans untouched in 60+ days. Plans should be either active or
  archived; long silence is drift.
- librarian_check_memory_drift.py: validates the user-memory dir
  (~/.claude/projects/<slug>/memory). Each MEMORY.md index line
  must point at a real file with frontmatter (name, description,
  type ∈ {user, feedback, project, reference}); no orphaned memory
  files. Memory dir is derived from repo root for portability.

Both run clean against current state.

Stage 1 declared complete. Promotable-archive and duplicate-prose
detection are moved to Stage 2 in the plan because both need LLM
judgment to avoid false positives — not pattern matching.

Next concrete step is the open Stage 1 finding from yesterday:
docs/performance.md at 914 lines, due for a split into sub-topic
notes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 23:44:25 +09:00
co-authored by Claude Opus 4.7
parent 5a1b944931
commit 6ecb233bdd
4 changed files with 220 additions and 10 deletions
+27 -10
View File
@@ -141,18 +141,35 @@ operator applies the patch.
in AGENTS.md. Caught one real finding on first run:
`docs/performance.md` at 914 lines — split into sub-topics deferred
as a separate task.
- ✅ Stage 1, piece 4: `scripts/librarian_check_stale_plans.py`. Uses
`git log -1 --format=%cs` per plan file; flags plans whose last
commit is older than 60 days. Clean on first run (all four plans
committed 2026-05-07).
- ✅ Stage 1, piece 5: `scripts/librarian_check_memory_drift.py`.
Validates that every MEMORY.md index line points to a real file
with required frontmatter fields (`name`, `description`, `type`
{user, feedback, project, reference}) and that no memory file is
orphaned from the index. Memory dir derived from repo root, so the
script is portable. Clean on first run.
## Stage 1 Remaining Checks
## Stage 1 Status: Complete
- Stale plans (mtime + git-log staleness heuristic).
- Promotable archive entries (deferred to Stage 2 — heuristic vs LLM
judgment is the open question).
- MEMORY.md drift (index lines vs target file `description:` frontmatter).
- Duplicate prose (high-overlap pairs across archive vs research).
All deterministic checks land. Two remaining concepts intentionally
moved out of Stage 1 because they require LLM judgment, not
deterministic detection:
- **Promotable archive entries** → Stage 2. "Durable conclusion"
detection is judgment, not pattern matching.
- **Duplicate prose** → Stage 2. Shingled-overlap heuristics produce
too many false positives in this repo's mix of archive snapshots
and derived research notes; LLM should decide whether two passages
are the *same idea* vs the *same evidence*.
Open Stage 1 finding to address: `docs/performance.md` at 914 lines.
## Next Concrete Step
Stale plan check (`scripts/librarian_check_stale_plans.py`). Walks
`docs/plans/*.md` and flags files whose mtime is older than N days
AND whose path hasn't appeared in `git log` over the same window.
This catches plans that drift out of mind without being archived.
Address the open Stage 1 finding by splitting `docs/performance.md`
into sub-topic notes under `docs/research/` (and dated archive
entries where appropriate). Sketch the split as a 1-page sub-plan
before doing the actual move so we don't shred a useful document.
+12
View File
@@ -27,6 +27,18 @@ if ! uv run python scripts/librarian_check_oversize.py; then
fi
echo
echo "→ Stale plans (no commit in 60 days)"
if ! uv run python scripts/librarian_check_stale_plans.py; then
overall=1
fi
echo
echo "→ MEMORY.md drift (index vs frontmatter)"
if ! uv run python scripts/librarian_check_memory_drift.py; then
overall=1
fi
echo
if [ "$overall" -eq 0 ]; then
echo "All Stage 1 checks passed."
else
+104
View File
@@ -0,0 +1,104 @@
"""MEMORY.md drift check for librarian Stage 1.
Validates the user-memory directory:
- Each MEMORY.md index line points to a real memory file.
- Each memory file has YAML frontmatter with required fields
(`name`, `description`, `type`) and a valid `type`.
- Reports orphan memory files (exist on disk but missing from index).
Memory dir is derived from the repo root, mirroring how Claude Code
resolves project-scoped memory paths. If the dir doesn't exist (fresh
checkout), the check exits 0 silently.
Usage:
uv run python scripts/librarian_check_memory_drift.py
"""
from __future__ import annotations
import re
from pathlib import Path
VALID_TYPES = {"user", "feedback", "project", "reference"}
REQUIRED_FIELDS = ("name", "description", "type")
INDEX_LINE = re.compile(r"^\s*-\s*\[([^\]]+)\]\(([^)]+)\)")
FRONTMATTER = re.compile(r"\A---\s*\n(.*?)\n---\s*\n", re.DOTALL)
def _repo_root() -> Path:
current = Path(__file__).resolve()
for parent in current.parents:
if (parent / "pyproject.toml").is_file():
return parent
raise RuntimeError("pyproject.toml을 찾을 수 없어 repository root를 판정할 수 없습니다.")
def _memory_dir(root: Path) -> Path:
project_slug = str(root).replace("/", "-")
return Path.home() / ".claude" / "projects" / project_slug / "memory"
def _parse_frontmatter(body: str) -> dict[str, str] | None:
m = FRONTMATTER.match(body)
if not m:
return None
fields: dict[str, str] = {}
for line in m.group(1).splitlines():
if ":" not in line:
continue
key, _, value = line.partition(":")
fields[key.strip()] = value.strip()
return fields
def main() -> int:
root = _repo_root()
mem_dir = _memory_dir(root)
index_file = mem_dir / "MEMORY.md"
if not index_file.is_file():
print(f"MEMORY.md not found at {mem_dir}; skipping memory check.")
return 0
findings: list[str] = []
referenced: set[str] = set()
for raw in index_file.read_text(encoding="utf-8").splitlines():
m = INDEX_LINE.match(raw)
if not m:
continue
target = m.group(2).strip()
referenced.add(target)
target_path = mem_dir / target
if not target_path.is_file():
findings.append(f"index points to missing file: {target}")
continue
body = target_path.read_text(encoding="utf-8")
fields = _parse_frontmatter(body)
if fields is None:
findings.append(f"{target}: missing or malformed frontmatter")
continue
for key in REQUIRED_FIELDS:
if key not in fields:
findings.append(f"{target}: frontmatter missing `{key}`")
if "type" in fields and fields["type"] not in VALID_TYPES:
findings.append(
f"{target}: invalid type `{fields['type']}` (allowed: {sorted(VALID_TYPES)})"
)
on_disk = {p.name for p in mem_dir.glob("*.md") if p.name != "MEMORY.md"}
for orphan in sorted(on_disk - referenced):
findings.append(f"orphan memory file (not in MEMORY.md index): {orphan}")
if not findings:
print("MEMORY.md and memory files consistent.")
return 0
print("MEMORY.md drift findings:")
for entry in findings:
print(f" {entry}")
return 1
if __name__ == "__main__":
raise SystemExit(main())
+77
View File
@@ -0,0 +1,77 @@
"""Stale plan check for librarian Stage 1.
Walks docs/plans/*.md (top-level only — archive subdir is separate)
and flags plans whose last git commit is older than STALE_DAYS.
Plans should be either actively worked on or archived (moved to
docs/plans/archive/) — long-untouched files are usually drift.
Usage:
uv run python scripts/librarian_check_stale_plans.py
"""
from __future__ import annotations
import subprocess
from datetime import datetime, timedelta
from pathlib import Path
STALE_DAYS = 60
def _repo_root() -> Path:
current = Path(__file__).resolve()
for parent in current.parents:
if (parent / "pyproject.toml").is_file():
return parent
raise RuntimeError("pyproject.toml을 찾을 수 없어 repository root를 판정할 수 없습니다.")
def _last_commit_date(root: Path, rel: Path) -> str | None:
result = subprocess.run(
["git", "log", "-1", "--format=%cs", "--", str(rel)],
capture_output=True,
text=True,
cwd=root,
check=False,
)
out = result.stdout.strip()
return out if out else None
def main() -> int:
root = _repo_root()
plans_dir = root / "docs" / "plans"
if not plans_dir.is_dir():
print("docs/plans not found; skipping stale-plan check.")
return 0
cutoff = (datetime.now() - timedelta(days=STALE_DAYS)).date()
stale: list[tuple[Path, str]] = []
for path in sorted(plans_dir.glob("*.md")):
rel = path.relative_to(root)
last = _last_commit_date(root, rel)
if last is None:
stale.append((rel, "untracked"))
continue
try:
last_date = datetime.strptime(last, "%Y-%m-%d").date()
except ValueError:
continue
if last_date < cutoff:
stale.append((rel, last))
if not stale:
print(f"No stale plans (all touched within {STALE_DAYS} days).")
return 0
print(f"Stale plan candidates (last commit > {STALE_DAYS} days ago):")
for rel, last in stale:
print(f" {last} {rel}")
print()
print("Action: archive (mv to docs/plans/archive/) or update if still active.")
return 1
if __name__ == "__main__":
raise SystemExit(main())