137 lines
5.1 KiB
Markdown
137 lines
5.1 KiB
Markdown
# Deep CFR Reproducibility Policy
|
|
|
|
**Last verified:** 2026-05-08, commit `0f85fa8`
|
|
**Source:** `docs/research/deep-cfr-reproducibility.md`
|
|
|
|
## Policy
|
|
|
|
Deep CFR experiments should target two different kinds of reproducibility:
|
|
|
|
1. **Debug reproducibility:** short runs should be bitwise reproducible on the
|
|
same machine when using deterministic settings.
|
|
2. **Research reproducibility:** reported conclusions should survive repeated
|
|
seeds and reasonable hardware differences, even when exact metric rows do
|
|
not match bit-for-bit.
|
|
|
|
Do not treat a single seeded multi-worker GPU run as a final research result.
|
|
Use it as an exploratory signal unless it is confirmed with matched seeds.
|
|
|
|
## Hardware Expectations
|
|
|
|
Exact same metrics are not guaranteed across machines, even with the same CUDA
|
|
version. GPU model, driver, PyTorch build, CPU scheduling, multiprocessing
|
|
timing, and floating-point reduction order can all affect exact trajectories.
|
|
|
|
Expected reliability by setting:
|
|
|
|
| Setting | Expected result |
|
|
| --- | --- |
|
|
| Same machine, same GPU, same code, `traversal.num_workers=1` | Best current option for bitwise debug checks |
|
|
| Same machine, same GPU, same code, multi-worker traversal | Current implementation target: bitwise stable across repeated runs |
|
|
| Same GPU/CUDA but different CPU | Current implementation target: same seed should follow the same logical trajectory |
|
|
| Different GPU or PyTorch/CUDA build | Exact equality is not expected; use multi-seed conclusions |
|
|
|
|
## Required Experiment Practice
|
|
|
|
For exploratory experiments:
|
|
|
|
- Single-seed runs are acceptable.
|
|
- Record the resolved config, run directory, W&B URL when used, and final eval
|
|
metrics.
|
|
- Label conclusions as provisional.
|
|
|
|
For claims worth keeping:
|
|
|
|
- Use matched seeds across baseline and treatment.
|
|
- Use at least 3 seeds; prefer 5 when runtime allows.
|
|
- Report mean and standard deviation for the key metrics.
|
|
- Keep `run.seed`, commit SHA, GPU, CUDA/PyTorch versions, and
|
|
`traversal.num_workers` visible in the run record.
|
|
|
|
Example matched-seed comparison:
|
|
|
|
```text
|
|
baseline: seeds 79, 80, 81
|
|
treatment: seeds 79, 80, 81
|
|
```
|
|
|
|
Compare the treatment against the baseline seed-by-seed, then report aggregate
|
|
statistics.
|
|
|
|
## Batch Size Interpretation
|
|
|
|
Not every batch size is a reproducibility-neutral setting.
|
|
|
|
Batching that should be algorithm-neutral under the deterministic target:
|
|
|
|
- traversal/interleaved policy inference batch size, such as
|
|
`traversal.interleave_max_batch`
|
|
- evaluation inference batch size, such as `evaluation.batch_size`
|
|
|
|
These settings should affect throughput, not the logical trajectory. Same code,
|
|
seed, hardware stack, and training config should produce the same non-timing
|
|
metrics when only these batching knobs change.
|
|
|
|
Batching that is an algorithmic training setting:
|
|
|
|
- `optimization.advantage_batch_size`
|
|
- `optimization.strategy_batch_size`
|
|
- update counts or optimizer schedule parameters
|
|
|
|
These settings change gradient estimates or optimizer updates. They are
|
|
experiment variables, not reproducibility-neutral execution settings. A run with
|
|
training mini-batch size 3 is not expected to match one with training mini-batch
|
|
size 7.
|
|
|
|
## Debug Mode
|
|
|
|
Use debug mode when checking exact reproducibility or suspected regressions:
|
|
|
|
- `traversal.num_workers=1`
|
|
- `evaluation.num_workers=1`
|
|
- PyTorch deterministic settings enabled:
|
|
- `torch.use_deterministic_algorithms(True)`
|
|
- `torch.backends.cudnn.benchmark = False`
|
|
- `torch.backends.cuda.matmul.allow_tf32 = False`
|
|
- `torch.backends.cudnn.allow_tf32 = False`
|
|
- short `run.max_iterations`, usually 3
|
|
- checkpoint writes disabled unless specifically needed
|
|
- W&B disabled
|
|
|
|
The expected check is to run the same command twice and compare non-timing
|
|
metrics in `metrics.jsonl`, including:
|
|
|
|
- `traversal/nodes`
|
|
- `memory/advantage`
|
|
- `memory/strategy`
|
|
- `loss/advantage`
|
|
- `loss/strategy`
|
|
- eval scores, if evaluation is enabled
|
|
|
|
Timing fields are not expected to match exactly.
|
|
|
|
## Implementation Priorities
|
|
|
|
Current implementation target:
|
|
|
|
1. Add a small reproducibility smoke script that runs two short debug jobs and
|
|
compares non-timing metrics.
|
|
2. Make worker count a performance setting, not an algorithm setting:
|
|
`traversal.num_workers=1`, `4`, and `8` should produce the same non-timing
|
|
metrics on the same code/config/seed.
|
|
3. Assign stable traversal IDs, derive RNG streams from traversal IDs, and merge
|
|
samples by traversal order rather than worker, batch, or completion order.
|
|
4. Stabilize interleaved scheduler request/context ordering so CPU scheduling
|
|
differences do not change the logical traversal path.
|
|
5. Enable PyTorch deterministic settings for reproducibility/debug runs. Keep
|
|
the speed impact explicit when comparing runtime.
|
|
6. Re-test same-seed runs across `traversal.num_workers=1`, `4`, and `8`.
|
|
7. Re-test on the same GPU/CUDA stack with a different CPU when available.
|
|
|
|
## Current Interpretation Rule
|
|
|
|
Until traversal-ID based deterministic scheduling and merge are implemented,
|
|
default multi-worker runs with the same seed may diverge. Interpret exact curves
|
|
cautiously. For research conclusions, prefer matched multi-seed comparisons over
|
|
bitwise row matching.
|