Commit Graph
4 Commits
Author SHA1 Message Date
coolguyandCodex fea8e5b9de measure julia flux mlp forward criterion
Co-Authored-By: Codex <codex@openai.com>
2026-05-07 21:27:34 +09:00
coolguyandClaude Opus 4.7 14dbf9c804 Add explicit pass/fail thresholds for Julia port criteria
Decision-in-advance for criteria 3 (multi-thread scaling), 4 (Flux/CUDA
MLP forward), and 5 (real-game-state slice). Each criterion lists PASS
/ PARTIAL / FAIL bands with concrete numerical thresholds, plus a
decision rule that maps {3, 4, 5} outcomes to a single action: port,
hybrid (Julia traversal + PyTorch networks), or stay on Python/Cython
and pursue Option B instead.

Cost-of-being-wrong asymmetry stated explicitly: port is months,
staying is zero work, so the GO bar is deliberately above 50% and the
STAY bar is permissive.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:12:10 +09:00
coolguyandCodex 100f498cf1 add heavy julia cfr thread scaling
Co-Authored-By: Codex <codex@openai.com>
2026-05-07 21:08:35 +09:00
coolguyandClaude Opus 4.7 ce9c6f6b93 Add Julia CFR-shape toy benchmark + Julia port evaluation thread
experiments/julia_cfr_toy/ ports a synthetic CFR external-sampling
traversal to both Julia and Cython for direct head-to-head
measurement of the actual project hot-path pattern (recursive
tree + mutable regret state + branch-heavy legal-action logic).

Headline result (2026-05-07, single run): Julia ~1.9× faster than
Cython on this pattern, 0 MB allocation, 0% GC time. Root regret
parity ε ≤ 1e-9. The GC-pause concern that was the main argument
against Julia adoption did not materialize. Cython's 21.3 MB
allocation suggests its implementation can be tightened, so the
honest gap window is roughly 1.3×–1.9×.

Multi-thread scaling (bench_cfr_threaded.jl): 2.44× wall-clock at 8T
but only 31% efficiency — inconclusive, likely a toy-size artifact
(per-thread workload too small to amortize dispatch). A heavier
per-thread workload sweep is the remaining decisive test.

docs/research/julia_port_evaluation.md captures this evidence
alongside the earlier safe-heuristic single-thread parity result
and lists the remaining decision criteria (multi-thread scaling with
heavier workload, Flux.jl+CUDA.jl coverage, real-game-state slice).
Do not commit to porting until multi-thread scaling is conclusively
settled.

ideas.md gets a second Active Research Threads pointer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 21:04:15 +09:00