coorl-lost-cities/docs/research/deep-cfr-v0-feature-parity.md

# Deep CFR v0: Subsystem Coverage vs. Legacy Reference

**Last verified:** 2026-05-07, commit `ad0be89`

Source: `docs/archive/deep-cfr-v0-gap-vs-coolrl.md`

## Question

What does this repository's Deep CFR implementation cover relative to the legacy
`../coolrl` reference, and where are the intentional gaps?

Short answer: **all core Deep CFR subsystems are implemented; the gaps are
tooling, not correctness.** Traversal, training, encoding, memory,
checkpoints, evaluation, self-play league, imitation pretraining, and PG
fine-tuning are all present in the Cython/PyTorch package. Missing items
relative to legacy (preset configs, per-worker observability, plotters, W&B)
are deliberately deferred. The real remaining frontier is *performance*
(Python-boundary policy calls, recursive Cython traversal), not feature parity.

## Scope

The goal of this repo from the start was *not* legacy feature parity — it was
higher training throughput via Cython hot-paths and cleaner experiment
infrastructure. The gap document tracks where parity has been achieved and where
it has not.

## What is fully implemented

**Traversal** — all of the core Deep CFR traversal logic lives in
`src/coolrl_lost_cities/games/classic/deep_cfr/traversal.pyx`:
recursive traversal with traverser/opponent node dispatch, outcome-sampling
with epsilon exploration and importance-weight correction, optional value
clipping, unsampled-regret modes (`zero` and `negative_node_value`), depth and
node-budget cutoffs with score-diff and rollout-based terminal values, deck-draw
chance sampling with push/pop state restoration, and instantaneous regret and
strategy memory collection.

The old Python fallback `traverser.py` has been removed from the mainline; Cython
is the sole traversal path.

**Training and memory** — `trainer.py`, `memory.py`, `networks.py` provide PyTorch
advantage networks (one per player) and a strategy network, legal-mask-aware
advantage loss, masked strategy cross-entropy, reservoir sampling with capacity
limits, single-process and multiprocess worker batches with result merging in the
parent.

**Encoding** — `encoding.pyx` exposes both a Python wrapper and a C-level buffer
write path. The feature set covers: phase flags, current/traversing player, deck
ratio, hand slot features, public expeditions for both players, public discards,
public card counts, total score and score diff, turn ratio, pending-discard
one-hot, and legal action mask.

**Runtime operations** — checkpoint save/load (`checkpoints.py`), config stored
in checkpoints and `config.json`, strategy-net policy adapter, evaluation against
registered classic bots (`evaluate.py`), training CLI and evaluation CLI
(`cli.py`), traversal benchmark CLI (`benchmark.py`), `metrics.jsonl` /
`runtime_progress.json` / `train.log` run artifacts, self-play league with
snapshot pool and weighted current/recent/older/anchor bucket sampling, safe-
heuristic anchor opponent, safe-heuristic imitation pretraining (`imitation.py`),
and policy-gradient fine-tuning (`policy_gradient.py`).

As of `ad0be89`, the package also includes `inference_server.py`,
`inference_client.py`, `inference_buffers.py`, `analyze.py`, `tracking.py`, and
`traversal_stats.py` — additions beyond the original plan that support batched
inference experiments and richer metrics collection.

## Intentional gaps (not blockers)

| Area | Legacy has | This repo | Notes |
|---|---|---|---|
| Config presets | Many experiments | Sparse YAML configs | Intentional — config-first, not preset-first |
| Multiprocess observability | Progress callbacks per worker, hotspot timing | Basic worker merge only | Low-priority tooling gap |
| Metrics visualization | Plot/status commands | `metrics.jsonl` only; no built-in plotter | `analyze.py` partially addresses this |
| Checkpoint artifacts | W&B integration | Local only | Not a correctness issue |
| Legacy visualization helpers | Present | Not ported | Not needed for training |

None of these gaps affect the correctness or usefulness of the training loop.

## Performance gap: the real remaining frontier

The original split from the legacy repo was motivated by traversal performance,
not feature parity. As of `ad0be89`:

- Game state mutation, legal-action generation, apply/undo, and cached scoring
  run in Cython (`game.pyx` in the parent package).
- Information-state encoding and regret matching have Cython modules
  (`encoding.pyx`, `cfr_math.pyx`).
- The full traversal loop runs through `traversal.pyx`.
- Policy inference and reservoir memory materialization still cross the Python
  boundary (PyTorch call and NumPy buffer write).
- Traversal is still recursive inside Cython; an explicit iterative Cython
  scheduler is a future optimization.

The batched inference path (`inference_server.py` / `inference_client.py`) is an
opt-in experiment toward reducing the Python boundary cost for policy calls, but
it is not the default training path.

The performance roadmap (C-level action enumeration, push/pop in tight loops,
batched memory writes, batched policy inference, and ultimately a Cython
iterative traversal scheduler) is the main remaining work, not legacy feature
parity.

## Practical implication

- Don't reach for the legacy repo to fill correctness gaps — there are none in
  scope. Reach for it only for tooling references (preset YAMLs, plotting,
  W&B wiring) where porting is deliberately deferred.
- New experiments should land in this repo's Cython traversal path; backporting
  to the legacy traverser is not a goal.
- When prioritizing optimization work, target the Python boundary
  (policy inference, NumPy buffer writes) before re-tuning anything already
  fully in Cython — that is where the remaining throughput is hiding.
- Treat tooling-gap items in the table above as "open tickets, not blockers";
  they should not gate training or eval work.