coorl-lost-cities/experiments/julia_flux_mlp/README.md

Julia Flux MLP Forward Benchmark

Criterion 4 for the Julia port evaluation.

Compares PyTorch DeepCFRMLP against a Flux/CUDA implementation with the same shape and identical exported weights:

• input dim: 365
• hidden size: 512
• hidden layers: 3
• output dim: 22
• activation: ReLU

Run:

uv run python experiments/julia_flux_mlp/bench_pytorch_vs_flux.py

The runner exports PyTorch weights and inputs to runs/tmp/, runs Flux on the same payload, checks output equivalence, then writes results.json.

Results (2026-05-07)

Host GPU: NVIDIA GeForce RTX 3090. Timing uses 10 warmup forwards, then 100 timed forwards, with CUDA synchronized around the timed loop in both runtimes.

Backend	batch	forward ms	μs/state	ratio vs PyTorch
PyTorch	1	0.0829	82.8755	1.00×
Flux	1	0.1669	166.8867	2.01×
PyTorch	64	0.0927	1.4477	1.00×
Flux	64	0.1909	2.9836	2.06×
PyTorch	256	0.0877	0.3427	1.00×
Flux	256	0.1758	0.6867	2.00×

Maximum output difference: 5.215e-08.

Criterion 4 threshold:

• bs=64 must be within ±20% of PyTorch.
• bs=1 and bs=256 must be within ±30% of PyTorch.

Verdict: FAIL. Flux/CUDA is ~2.0× slower than PyTorch at all measured batch sizes for this model shape.