Add batched traversal inference server (Option A) behind opt-in flag

Implements the central inference server pattern: a dedicated GPU
process owns advantage/strategy/league networks, batches policy
requests across traversal workers via shared-memory tensor pool, and
returns logits. Workers route forward calls through InferenceClient /
NetworkProxy when traversal.inference_backend == "server".

Default remains traversal.inference_backend: local. The server
backend regresses iter time ~3.8× on the inspected default config
(small-model dispatch + sync-blocking traversal capping realized
batch at ~num_workers=8 instead of the bs=64-256 needed to amortize
IPC overhead). Keeping the implementation behind the flag lets us
re-enable when (a) model size grows, (b) per-worker interleaved
traversal lands, or (c) eval becomes dominant — see
docs/performance.md "Option A Bench Result and Structural Ceiling"
for the full diagnosis.

Plumbing included:
- inference_buffers.py: shared-memory tensor pool with slot
  management.
- inference_client.py: per-worker client + NetworkProxy adapter for
  the existing traversal.pyx call sites.
- inference_server.py: spawn-context server process with
  batch-window aggregation, weight sync, shutdown sentinel.
- bench_inference_backend.py: A/B between local and server backends
  with eval/checkpoint disabled.
- test_inference_server.py: round-trip and integration tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 20:05:16 +09:00
co-authored by Claude Opus 4.7
parent 05de0e2a81
commit a7ab94e096
10 changed files with 1375 additions and 79 deletions
+108
View File
@@ -0,0 +1,108 @@
run:
experiment_name: deep-cfr-default-server
seed: 79
max_iterations: 1000
max_minutes: null
device: cuda
use_amp: false
rules:
n_colors: 5
n_ranks: 9
min_rank: 2
n_handshakes: 3
hand_size: 8
expedition_penalty: -20
bonus_threshold: 8
bonus_amount: 20
encoding:
derived_playability: true
slot_aware_playability: true
network:
hidden_size: 512
num_layers: 3
activation: relu
traversal:
traversals_per_player: 280
max_depth: null
max_nodes_per_traversal: 1000
regret_matching_epsilon: 0.0001
outcome_sampling_epsilon: 0.2
outcome_sampling_value_clip: 500.0
outcome_unsampled_regret: zero
cutoff_value_mode: score_diff
cutoff_rollouts: 0
cutoff_rollout_policy: random
cutoff_rollout_max_steps: 300
opponent_policy: average_strategy
strategy_sample_interval: 1
store_strategy_on_traverser_nodes: true
store_strategy_on_opponent_nodes: false
num_workers: 8
worker_chunk_size: 8
progress_every_traversals: 10
endpoint_depth_bucket_width: 100
endpoint_depth_bucket_max: 1000
inference_backend: server
regret_matching:
all_negative_fallback: argmax_tiebreak
training_weighting:
mode: lcfr
self_play:
snapshot_every: 1
max_snapshots: 0
anchor_probability: 0.0
current_weight: 1.0
recent_weight: 0.0
older_weight: 0.0
anchor_weight: 0.0
recent_window: 5
optimization:
advantage_updates_per_iteration: 512
strategy_updates_per_iteration: 512
advantage_batch_size: 1024
strategy_batch_size: 1024
learning_rate: 1.0e-4
weight_decay: 0.0001
grad_clip: 1.0
memory:
advantage_capacity: 2000000
strategy_capacity: 2000000
checkpoint:
save_latest: true
save_every: 100
progress_interval_seconds: 20.0
exact_resume: false
evaluation:
eval_every: 25
games: 100
opponents:
- random
- passive_discard
- safe_heuristic
- safe_heuristic_loose
- safe_heuristic_strict
- noisy_safe
max_steps: 10000
on_max_steps: score_diff
batch_size: 64
device: trainer
num_workers: 4
inference_server:
device: cuda
num_slots: null
max_batch: 256
batch_window_us: 200
weight_sync_every: 1
use_amp: false