coolguyandClaude Opus 4.7
004b913a7b
Three coordinated hygiene changes; none target the diagnosed
selection-bias bottleneck. They make the codebase honestly reflect the
pure-self-play stance and reduce dashboard noise.
Bot rename (drop the unhelpful safe_ prefix; suffixes describe behaviour):
- safe_heuristic_loose -> heuristic_aggressive
- safe_heuristic -> heuristic_balanced
- safe_heuristic_strict -> heuristic_cautious
- noisy_safe -> heuristic_noisy
- passive_discard -> discard_only
Class renames in bots/: SafeHeuristicBot -> HeuristicBot,
SafeHeuristicParams -> HeuristicParams, PassiveDiscardBot -> DiscardOnlyBot,
plus loose/strict parameter constants. Backwards compatibility was dropped
intentionally per user instruction; no aliases. Active configs, docs,
scripts, tests updated. Archive directories (configs/archive,
docs/archive, runs/archive) left intact and may still reference old
names per their read-only policy. The src/.../bots/passive.py module was
renamed to discard_only.py via git mv.
Analyze plot curation (deep_cfr/analyze.py):
- Added analysis_00_core.png as the canonical daily dashboard with 10
heuristic-free metrics (loss/{advantage,strategy}; vs heuristic_cautious:
avg_score_diff0, win_rate0, avg_opened_colors, positive_expedition_rate,
bonus_expedition_rate, score_per_opened_color, policy_entropy; vs random:
win_rate0).
- Removed analysis_05_open_quality.png (bad/weak/good open rates,
recoverable score) and analysis_07_calibration.png (calibration gap,
recoverable mean) - both relied on the heuristic recoverable_score
classifier already dropped from inputs.
- Removed SELECTIVITY_PLOTS and plot_selectivity (heuristic-laden).
- SUMMARY_EVAL_METRICS no longer includes bad_open_rate or
calibration_gap.
- PlotSpec gained an opponents allowlist so the new core section can pin
a specific opponent per panel without restructuring plot_section.
Tiered evaluation cadence (EvaluationConfig):
- Added extended_opponents and extended_eval_every (default 0 = disabled).
- opponents_for_iteration(iteration) returns the core list every
eval_every and appends extended_opponents (de-duplicated) when
iteration is also a multiple of extended_eval_every.
- default.yaml now uses 3 core opponents (random, discard_only,
heuristic_cautious) every 5 iterations and 3 extended opponents
(heuristic_balanced, heuristic_aggressive, heuristic_noisy) every 50
iterations. random is the floor sanity. discard_only is the
zero-pit detector / absolute-score reference (its score is always 0,
so eval/discard_only/avg_score_diff0 directly equals the model's raw
average score). heuristic_cautious is the ceiling and the
archive-comparable benchmark used in the prior diagnostic sections.
Net eval cost reduction: roughly 50% (3 opponents x every 5 iter, plus
6 opponents x every 50 iter, vs the prior 6 x every 5).
Documented in docs/plans/deep-cfr-selectivity.md section 9.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3.1 KiB
Batched and Parallel Evaluation in Deep CFR
Last verified: 2026-05-08, commit b0b3855
Source: docs/archive/deep-cfr-batched-evaluation-2026-05-07.md
Question
Evaluation of Deep CFR strategies against heuristic opponents can be a major bottleneck during training, especially when using CUDA for network inference. How can batched inference and parallel execution be leveraged to minimize this cost without introducing synchronization overhead?
Code reference
src/coolrl_lost_cities/games/classic/deep_cfr/evaluate.py,StrategyNetPolicy.select_actions_batch(line 174): Implements batched policy network inference, allowing multiple games to share a single GPU forward pass.src/coolrl_lost_cities/games/classic/deep_cfr/evaluate.py(line 220): Performs batched entropy calculation directly on the GPU using Torch tensors.src/coolrl_lost_cities/games/classic/deep_cfr/trainer.py,DeepCFRTrainer._evaluate_parallel(line 891): Orchestrates parallel evaluation across different opponents usingProcessPoolExecutor.
Analysis
The primary bottleneck in CUDA-based evaluation is the overhead of launching small GPU kernels for single-state network inference. By batching evaluation games, we can saturate the GPU's compute units more effectively. Empirical results from May 2026 show that increasing the evaluation batch size to 64 reduced evaluation time from approximately 61.8 seconds to 14.8 seconds per iteration.
A critical refinement in the batched implementation was the handling of policy entropy. Initial versions that calculated entropy per-row on the CPU incurred significant synchronization penalties because each row required a GPU-to-CPU transfer. Moving the entropy calculation into the Torch post-processing pipeline—specifically calculating it directly on the probs_tensor (line 220)—ensures that the computation remains on the device and only the final results are transferred back to the host in bulk.
Once network inference is batched, the remaining bottleneck often shifts to the CPU-bound logic of heuristic opponents (e.g., heuristic_cautious). Parallelizing the evaluation across multiple workers allows the trainer to evaluate against multiple opponents simultaneously. For a single iteration profile, using 4 parallel workers reduced wall-clock evaluation time from 14.8 seconds to 6.4 seconds, achieving a ~2.3x speedup.
Practical implication
- Enable Batching: For GPU-accelerated training, always configure
evaluation.batch_size(typically 64 or 128) to minimize kernel launch overhead and maximize throughput. - Consolidate Device Operations: Keep post-inference operations (legal masking, softmax, entropy) in Torch tensors to avoid blocking the GPU with frequent host-device synchronizations.
- Parallelize Opponents: Set
evaluation.num_workersto match the number of opponents being evaluated (up to the available CPU cores). This effectively hides the latency of slower heuristic bots behind the network inference of others.
References
docs/archive/deep-cfr-batched-evaluation-2026-05-07.mddocs/performance.md