Add batched traversal inference server (Option A) behind opt-in flag

Implements the central inference server pattern: a dedicated GPU
process owns advantage/strategy/league networks, batches policy
requests across traversal workers via shared-memory tensor pool, and
returns logits. Workers route forward calls through InferenceClient /
NetworkProxy when traversal.inference_backend == "server".

Default remains traversal.inference_backend: local. The server
backend regresses iter time ~3.8× on the inspected default config
(small-model dispatch + sync-blocking traversal capping realized
batch at ~num_workers=8 instead of the bs=64-256 needed to amortize
IPC overhead). Keeping the implementation behind the flag lets us
re-enable when (a) model size grows, (b) per-worker interleaved
traversal lands, or (c) eval becomes dominant — see
docs/performance.md "Option A Bench Result and Structural Ceiling"
for the full diagnosis.

Plumbing included:
- inference_buffers.py: shared-memory tensor pool with slot
  management.
- inference_client.py: per-worker client + NetworkProxy adapter for
  the existing traversal.pyx call sites.
- inference_server.py: spawn-context server process with
  batch-window aggregation, weight sync, shutdown sentinel.
- bench_inference_backend.py: A/B between local and server backends
  with eval/checkpoint disabled.
- test_inference_server.py: round-trip and integration tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-07 20:05:16 +09:00
co-authored by Claude Opus 4.7
parent 05de0e2a81
commit a7ab94e096
10 changed files with 1375 additions and 79 deletions
+9
View File
@@ -46,6 +46,7 @@ traversal:
progress_every_traversals: 10
endpoint_depth_bucket_width: 100
endpoint_depth_bucket_max: 1000
inference_backend: local
regret_matching:
all_negative_fallback: argmax_tiebreak
@@ -97,3 +98,11 @@ evaluation:
batch_size: 64
device: trainer
num_workers: 4
inference_server:
device: cuda
num_slots: null
max_batch: 256
batch_window_us: 200
weight_sync_every: 1
use_amp: false