Implements the central inference server pattern: a dedicated GPU
process owns advantage/strategy/league networks, batches policy
requests across traversal workers via shared-memory tensor pool, and
returns logits. Workers route forward calls through InferenceClient /
NetworkProxy when traversal.inference_backend == "server".
Default remains traversal.inference_backend: local. The server
backend regresses iter time ~3.8× on the inspected default config
(small-model dispatch + sync-blocking traversal capping realized
batch at ~num_workers=8 instead of the bs=64-256 needed to amortize
IPC overhead). Keeping the implementation behind the flag lets us
re-enable when (a) model size grows, (b) per-worker interleaved
traversal lands, or (c) eval becomes dominant — see
docs/performance.md "Option A Bench Result and Structural Ceiling"
for the full diagnosis.
Plumbing included:
- inference_buffers.py: shared-memory tensor pool with slot
management.
- inference_client.py: per-worker client + NetworkProxy adapter for
the existing traversal.pyx call sites.
- inference_server.py: spawn-context server process with
batch-window aggregation, weight sync, shutdown sentinel.
- bench_inference_backend.py: A/B between local and server backends
with eval/checkpoint disabled.
- test_inference_server.py: round-trip and integration tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
coolguyandClaude Opus 4.7
a7ab94e096