coorl-lost-cities/docs/research/deep-cfr-evaluation-profile-plan.md

Deep CFR Evaluation Profiling

Last verified: 2026-05-08, commit 5c221fb Source: docs/archive/deep-cfr-evaluation-profile-plan.md

Question

How are evaluation runtime costs categorized in Deep CFR, and what do these metrics reveal about system bottlenecks?

Evaluation is a critical path for measuring agent progress, but its runtime can be unpredictable. To move beyond wall-clock guessing, the system instruments the evaluation loop with granular counters that distinguish between neural network inference, state encoding, and game engine overhead.

Code reference

The primary instrumentation structure is the EvalRuntimeCounters dataclass in src/coolrl_lost_cities/games/classic/deep_cfr/evaluate.py (line 20). It tracks elapsed seconds across several distinct phases of a single game step:

@dataclass
class EvalRuntimeCounters:
    policy_turns: int = 0
    opponent_turns: int = 0
    policy_select_seconds: float = 0.0
    policy_legal_mask_seconds: float = 0.0
    policy_encoding_seconds: float = 0.0
    policy_network_seconds: float = 0.0
    policy_postprocess_seconds: float = 0.0
    opponent_act_seconds: float = 0.0
    apply_action_seconds: float = 0.0
    diagnostics_seconds: float = 0.0
    final_scoring_seconds: float = 0.0

These metrics are updated in select_actions and action_distribution (lines 200-280), capturing the micro-timing of every policy request.

Performance Analysis

The instrumentation allows for a tiered analysis of the evaluation bottleneck. By comparing these counters, one can pinpoint the specific layer responsible for performance degradation:

1. The Policy Path (policy_select_seconds)

This is the total time spent by the agent under evaluation. It is further subdivided to identify efficiency gaps in the neural pipeline:

• policy_network_seconds: Time spent inside the PyTorch forward pass. If this dominates, the bottleneck is model inference. For small models on CUDA, this often signals high kernel launch overhead for batch-size-1 requests.
• policy_encoding_seconds: Time spent converting GameState objects into numerical info-state tensors. High values here suggest that the Python-based feature engineering is a bottleneck.
• policy_legal_mask_seconds: Time spent calculating legal moves. In Lost Cities, this involves scanning the hand and board state.

2. Environment and Opponents

• opponent_act_seconds: Time spent by the opponent bot. When evaluating against expensive bots (like heuristic-heavy search agents), this metric isolates their cost from the main agent's performance.
• apply_action_seconds: The cost of the game engine itself (GameState.apply_action). High values indicate that the Cython game logic is the primary constraint.

Interpretation

The relationship between these metrics dictates the optimization strategy. If policy_network_seconds is the primary driver, the system is "model-bound," and improvements should focus on batching evaluation games or using inference accelerators like TensorRT. Conversely, if policy_encoding_seconds dominates, the system is "feature-bound," and the feature extraction logic should be moved to Cython or vectorized.

When opponent_act_seconds dominates, any local optimizations to the strategy network or encoding will have negligible impact on total evaluation time, as the bottleneck resides in the external bot's implementation.

Practical Implications

• Optimization Priority: Always check the ratio of policy_network_seconds to policy_select_seconds before attempting model optimizations.
• Device Selection: Large policy_network_seconds on CUDA relative to CPU for small models is a known symptom of launch-latency saturation, justifying a move to CPU for serial evaluation.
• Regression Testing: Evaluation metrics should be compared across iterations (e.g., comparing iteration 5 vs 10) to detect memory leaks or data structure bloat in the diagnostics path (diagnostics_seconds).

References

• docs/research/deep-cfr-evaluation-profile.md (Analysis of CUDA vs CPU latency)
• src/coolrl_lost_cities/games/classic/deep_cfr/evaluate.py (Implementation)