Complete librarian Stage 1: stale plans + MEMORY drift
Adds the last two deterministic Stage 1 checks and wires them into
scripts/librarian.sh:
- librarian_check_stale_plans.py: per docs/plans/*.md, uses
`git log -1 --format=%cs` to get the last commit date and flags
plans untouched in 60+ days. Plans should be either active or
archived; long silence is drift.
- librarian_check_memory_drift.py: validates the user-memory dir
(~/.claude/projects/<slug>/memory). Each MEMORY.md index line
must point at a real file with frontmatter (name, description,
type ∈ {user, feedback, project, reference}); no orphaned memory
files. Memory dir is derived from repo root for portability.
Both run clean against current state.
Stage 1 declared complete. Promotable-archive and duplicate-prose
detection are moved to Stage 2 in the plan because both need LLM
judgment to avoid false positives — not pattern matching.
Next concrete step is the open Stage 1 finding from yesterday:
docs/performance.md at 914 lines, due for a split into sub-topic
notes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
+27
-10
@@ -141,18 +141,35 @@ operator applies the patch.
|
|||||||
in AGENTS.md. Caught one real finding on first run:
|
in AGENTS.md. Caught one real finding on first run:
|
||||||
`docs/performance.md` at 914 lines — split into sub-topics deferred
|
`docs/performance.md` at 914 lines — split into sub-topics deferred
|
||||||
as a separate task.
|
as a separate task.
|
||||||
|
- ✅ Stage 1, piece 4: `scripts/librarian_check_stale_plans.py`. Uses
|
||||||
|
`git log -1 --format=%cs` per plan file; flags plans whose last
|
||||||
|
commit is older than 60 days. Clean on first run (all four plans
|
||||||
|
committed 2026-05-07).
|
||||||
|
- ✅ Stage 1, piece 5: `scripts/librarian_check_memory_drift.py`.
|
||||||
|
Validates that every MEMORY.md index line points to a real file
|
||||||
|
with required frontmatter fields (`name`, `description`, `type` ∈
|
||||||
|
{user, feedback, project, reference}) and that no memory file is
|
||||||
|
orphaned from the index. Memory dir derived from repo root, so the
|
||||||
|
script is portable. Clean on first run.
|
||||||
|
|
||||||
## Stage 1 Remaining Checks
|
## Stage 1 Status: Complete
|
||||||
|
|
||||||
- Stale plans (mtime + git-log staleness heuristic).
|
All deterministic checks land. Two remaining concepts intentionally
|
||||||
- Promotable archive entries (deferred to Stage 2 — heuristic vs LLM
|
moved out of Stage 1 because they require LLM judgment, not
|
||||||
judgment is the open question).
|
deterministic detection:
|
||||||
- MEMORY.md drift (index lines vs target file `description:` frontmatter).
|
|
||||||
- Duplicate prose (high-overlap pairs across archive vs research).
|
- **Promotable archive entries** → Stage 2. "Durable conclusion"
|
||||||
|
detection is judgment, not pattern matching.
|
||||||
|
- **Duplicate prose** → Stage 2. Shingled-overlap heuristics produce
|
||||||
|
too many false positives in this repo's mix of archive snapshots
|
||||||
|
and derived research notes; LLM should decide whether two passages
|
||||||
|
are the *same idea* vs the *same evidence*.
|
||||||
|
|
||||||
|
Open Stage 1 finding to address: `docs/performance.md` at 914 lines.
|
||||||
|
|
||||||
## Next Concrete Step
|
## Next Concrete Step
|
||||||
|
|
||||||
Stale plan check (`scripts/librarian_check_stale_plans.py`). Walks
|
Address the open Stage 1 finding by splitting `docs/performance.md`
|
||||||
`docs/plans/*.md` and flags files whose mtime is older than N days
|
into sub-topic notes under `docs/research/` (and dated archive
|
||||||
AND whose path hasn't appeared in `git log` over the same window.
|
entries where appropriate). Sketch the split as a 1-page sub-plan
|
||||||
This catches plans that drift out of mind without being archived.
|
before doing the actual move so we don't shred a useful document.
|
||||||
|
|||||||
@@ -27,6 +27,18 @@ if ! uv run python scripts/librarian_check_oversize.py; then
|
|||||||
fi
|
fi
|
||||||
echo
|
echo
|
||||||
|
|
||||||
|
echo "→ Stale plans (no commit in 60 days)"
|
||||||
|
if ! uv run python scripts/librarian_check_stale_plans.py; then
|
||||||
|
overall=1
|
||||||
|
fi
|
||||||
|
echo
|
||||||
|
|
||||||
|
echo "→ MEMORY.md drift (index vs frontmatter)"
|
||||||
|
if ! uv run python scripts/librarian_check_memory_drift.py; then
|
||||||
|
overall=1
|
||||||
|
fi
|
||||||
|
echo
|
||||||
|
|
||||||
if [ "$overall" -eq 0 ]; then
|
if [ "$overall" -eq 0 ]; then
|
||||||
echo "All Stage 1 checks passed."
|
echo "All Stage 1 checks passed."
|
||||||
else
|
else
|
||||||
|
|||||||
@@ -0,0 +1,104 @@
|
|||||||
|
"""MEMORY.md drift check for librarian Stage 1.
|
||||||
|
|
||||||
|
Validates the user-memory directory:
|
||||||
|
- Each MEMORY.md index line points to a real memory file.
|
||||||
|
- Each memory file has YAML frontmatter with required fields
|
||||||
|
(`name`, `description`, `type`) and a valid `type`.
|
||||||
|
- Reports orphan memory files (exist on disk but missing from index).
|
||||||
|
|
||||||
|
Memory dir is derived from the repo root, mirroring how Claude Code
|
||||||
|
resolves project-scoped memory paths. If the dir doesn't exist (fresh
|
||||||
|
checkout), the check exits 0 silently.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
uv run python scripts/librarian_check_memory_drift.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
VALID_TYPES = {"user", "feedback", "project", "reference"}
|
||||||
|
REQUIRED_FIELDS = ("name", "description", "type")
|
||||||
|
|
||||||
|
INDEX_LINE = re.compile(r"^\s*-\s*\[([^\]]+)\]\(([^)]+)\)")
|
||||||
|
FRONTMATTER = re.compile(r"\A---\s*\n(.*?)\n---\s*\n", re.DOTALL)
|
||||||
|
|
||||||
|
|
||||||
|
def _repo_root() -> Path:
|
||||||
|
current = Path(__file__).resolve()
|
||||||
|
for parent in current.parents:
|
||||||
|
if (parent / "pyproject.toml").is_file():
|
||||||
|
return parent
|
||||||
|
raise RuntimeError("pyproject.toml을 찾을 수 없어 repository root를 판정할 수 없습니다.")
|
||||||
|
|
||||||
|
|
||||||
|
def _memory_dir(root: Path) -> Path:
|
||||||
|
project_slug = str(root).replace("/", "-")
|
||||||
|
return Path.home() / ".claude" / "projects" / project_slug / "memory"
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_frontmatter(body: str) -> dict[str, str] | None:
|
||||||
|
m = FRONTMATTER.match(body)
|
||||||
|
if not m:
|
||||||
|
return None
|
||||||
|
fields: dict[str, str] = {}
|
||||||
|
for line in m.group(1).splitlines():
|
||||||
|
if ":" not in line:
|
||||||
|
continue
|
||||||
|
key, _, value = line.partition(":")
|
||||||
|
fields[key.strip()] = value.strip()
|
||||||
|
return fields
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
root = _repo_root()
|
||||||
|
mem_dir = _memory_dir(root)
|
||||||
|
index_file = mem_dir / "MEMORY.md"
|
||||||
|
if not index_file.is_file():
|
||||||
|
print(f"MEMORY.md not found at {mem_dir}; skipping memory check.")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
findings: list[str] = []
|
||||||
|
referenced: set[str] = set()
|
||||||
|
|
||||||
|
for raw in index_file.read_text(encoding="utf-8").splitlines():
|
||||||
|
m = INDEX_LINE.match(raw)
|
||||||
|
if not m:
|
||||||
|
continue
|
||||||
|
target = m.group(2).strip()
|
||||||
|
referenced.add(target)
|
||||||
|
target_path = mem_dir / target
|
||||||
|
if not target_path.is_file():
|
||||||
|
findings.append(f"index points to missing file: {target}")
|
||||||
|
continue
|
||||||
|
body = target_path.read_text(encoding="utf-8")
|
||||||
|
fields = _parse_frontmatter(body)
|
||||||
|
if fields is None:
|
||||||
|
findings.append(f"{target}: missing or malformed frontmatter")
|
||||||
|
continue
|
||||||
|
for key in REQUIRED_FIELDS:
|
||||||
|
if key not in fields:
|
||||||
|
findings.append(f"{target}: frontmatter missing `{key}`")
|
||||||
|
if "type" in fields and fields["type"] not in VALID_TYPES:
|
||||||
|
findings.append(
|
||||||
|
f"{target}: invalid type `{fields['type']}` (allowed: {sorted(VALID_TYPES)})"
|
||||||
|
)
|
||||||
|
|
||||||
|
on_disk = {p.name for p in mem_dir.glob("*.md") if p.name != "MEMORY.md"}
|
||||||
|
for orphan in sorted(on_disk - referenced):
|
||||||
|
findings.append(f"orphan memory file (not in MEMORY.md index): {orphan}")
|
||||||
|
|
||||||
|
if not findings:
|
||||||
|
print("MEMORY.md and memory files consistent.")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
print("MEMORY.md drift findings:")
|
||||||
|
for entry in findings:
|
||||||
|
print(f" {entry}")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
"""Stale plan check for librarian Stage 1.
|
||||||
|
|
||||||
|
Walks docs/plans/*.md (top-level only — archive subdir is separate)
|
||||||
|
and flags plans whose last git commit is older than STALE_DAYS.
|
||||||
|
Plans should be either actively worked on or archived (moved to
|
||||||
|
docs/plans/archive/) — long-untouched files are usually drift.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
uv run python scripts/librarian_check_stale_plans.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
STALE_DAYS = 60
|
||||||
|
|
||||||
|
|
||||||
|
def _repo_root() -> Path:
|
||||||
|
current = Path(__file__).resolve()
|
||||||
|
for parent in current.parents:
|
||||||
|
if (parent / "pyproject.toml").is_file():
|
||||||
|
return parent
|
||||||
|
raise RuntimeError("pyproject.toml을 찾을 수 없어 repository root를 판정할 수 없습니다.")
|
||||||
|
|
||||||
|
|
||||||
|
def _last_commit_date(root: Path, rel: Path) -> str | None:
|
||||||
|
result = subprocess.run(
|
||||||
|
["git", "log", "-1", "--format=%cs", "--", str(rel)],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
cwd=root,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
out = result.stdout.strip()
|
||||||
|
return out if out else None
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
root = _repo_root()
|
||||||
|
plans_dir = root / "docs" / "plans"
|
||||||
|
if not plans_dir.is_dir():
|
||||||
|
print("docs/plans not found; skipping stale-plan check.")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
cutoff = (datetime.now() - timedelta(days=STALE_DAYS)).date()
|
||||||
|
|
||||||
|
stale: list[tuple[Path, str]] = []
|
||||||
|
for path in sorted(plans_dir.glob("*.md")):
|
||||||
|
rel = path.relative_to(root)
|
||||||
|
last = _last_commit_date(root, rel)
|
||||||
|
if last is None:
|
||||||
|
stale.append((rel, "untracked"))
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
last_date = datetime.strptime(last, "%Y-%m-%d").date()
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
if last_date < cutoff:
|
||||||
|
stale.append((rel, last))
|
||||||
|
|
||||||
|
if not stale:
|
||||||
|
print(f"No stale plans (all touched within {STALE_DAYS} days).")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
print(f"Stale plan candidates (last commit > {STALE_DAYS} days ago):")
|
||||||
|
for rel, last in stale:
|
||||||
|
print(f" {last} {rel}")
|
||||||
|
print()
|
||||||
|
print("Action: archive (mv to docs/plans/archive/) or update if still active.")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
Reference in New Issue
Block a user