coorl-lost-cities/docs/research/deep-cfr-reproducibility-policy.md

Deep CFR Reproducibility Policy

Last verified: 2026-05-08, commit 0f85fa8 Source: docs/research/deep-cfr-reproducibility.md

Policy

Deep CFR experiments should target two different kinds of reproducibility:

1. Debug reproducibility: short runs should be bitwise reproducible on the same machine when using deterministic settings.
2. Research reproducibility: reported conclusions should survive repeated seeds and reasonable hardware differences, even when exact metric rows do not match bit-for-bit.

Do not treat a single seeded multi-worker GPU run as a final research result. Use it as an exploratory signal unless it is confirmed with matched seeds.

Hardware Expectations

Exact same metrics are not guaranteed across machines, even with the same CUDA version. GPU model, driver, PyTorch build, CPU scheduling, multiprocessing timing, and floating-point reduction order can all affect exact trajectories.

Expected reliability by setting:

Setting	Expected result
Same machine, same GPU, same code, traversal.num_workers=1	Best current option for bitwise debug checks
Same machine, same GPU, same code, multi-worker traversal	Current implementation target: bitwise stable across repeated runs
Same GPU/CUDA but different CPU	Current implementation target: same seed should follow the same logical trajectory
Different GPU or PyTorch/CUDA build	Exact equality is not expected; use multi-seed conclusions

Required Experiment Practice

For exploratory experiments:

• Single-seed runs are acceptable.
• Record the resolved config, run directory, W&B URL when used, and final eval metrics.
• Label conclusions as provisional.

For claims worth keeping:

• Use matched seeds across baseline and treatment.
• Use at least 3 seeds; prefer 5 when runtime allows.
• Report mean and standard deviation for the key metrics.
• Keep run.seed, commit SHA, GPU, CUDA/PyTorch versions, and traversal.num_workers visible in the run record.

Example matched-seed comparison:

baseline:  seeds 79, 80, 81
treatment: seeds 79, 80, 81

Compare the treatment against the baseline seed-by-seed, then report aggregate statistics.

Batch Size Interpretation

Not every batch size is a reproducibility-neutral setting.

Batching that should be algorithm-neutral under the deterministic target:

• traversal/interleaved policy inference batch size, such as traversal.interleave_max_batch
• evaluation inference batch size, such as evaluation.batch_size

These settings should affect throughput, not the logical trajectory. Same code, seed, hardware stack, and training config should produce the same non-timing metrics when only these batching knobs change.

Batching that is an algorithmic training setting:

• optimization.advantage_batch_size
• optimization.strategy_batch_size
• update counts or optimizer schedule parameters

These settings change gradient estimates or optimizer updates. They are experiment variables, not reproducibility-neutral execution settings. A run with training mini-batch size 3 is not expected to match one with training mini-batch size 7.

Debug Mode

Use debug mode when checking exact reproducibility or suspected regressions:

• traversal.num_workers=1
• evaluation.num_workers=1
• PyTorch deterministic settings enabled:
  • torch.use_deterministic_algorithms(True)
  • torch.backends.cudnn.benchmark = False
  • torch.backends.cuda.matmul.allow_tf32 = False
  • torch.backends.cudnn.allow_tf32 = False
• short run.max_iterations, usually 3
• checkpoint writes disabled unless specifically needed
• W&B disabled

The expected check is to run the same command twice and compare non-timing metrics in metrics.jsonl, including:

• traversal/nodes
• memory/advantage
• memory/strategy
• loss/advantage
• loss/strategy
• eval scores, if evaluation is enabled

Timing fields are not expected to match exactly.

Implementation Priorities

Current implementation target:

1. Add a small reproducibility smoke script that runs two short debug jobs and compares non-timing metrics.
2. Make worker count a performance setting, not an algorithm setting: traversal.num_workers=1, 4, and 8 should produce the same non-timing metrics on the same code/config/seed.
3. Assign stable traversal IDs, derive RNG streams from traversal IDs, and merge samples by traversal order rather than worker, batch, or completion order.
4. Stabilize interleaved scheduler request/context ordering so CPU scheduling differences do not change the logical traversal path.
5. Enable PyTorch deterministic settings for reproducibility/debug runs. Keep the speed impact explicit when comparing runtime.
6. Re-test same-seed runs across traversal.num_workers=1, 4, and 8.
7. Re-test on the same GPU/CUDA stack with a different CPU when available.

Current Interpretation Rule

Until traversal-ID based deterministic scheduling and merge are implemented, default multi-worker runs with the same seed may diverge. Interpret exact curves cautiously. For research conclusions, prefer matched multi-seed comparisons over bitwise row matching.