JAX PPO Ladder v2 Report - 2026-07-05

Status: PASS for the ordered ladder gates; canary warnings remain. Hardware: RTX 3090 via uv run --with 'jax[cuda12]'. Artifact roots:

Expert bot: /mnt/2tbhdd/coolrl-lost-cities-artifacts/expert-bot/2026-07-05/
Ladder v2: /mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/

Protocol

Part A added a pure-JAX heuristic_expert opponent and evaluated it with a fixed shuffle bank plus duplicate play. Part B trained PPO from random initialization for each gate with the same 250-update configuration used by the original ladder, changing only the opponent and artifact root:

Gate 1: discard_only
Gate 2: heuristic_balanced
Gate 3: heuristic_expert

All evaluations below used 10,000 deck orders with duplicate seat-swapped play, for 20,000 evaluated games.

Part A - Expert Bot Gate

Check	Result	Win rate	Mean diff	CI95 diff	Length	Opened colors	Play rate
Expert mirror, 1,000 duplicate pairs	PASS	0.4815	0.0000	[-0.6956, +0.6956]	45.1560	2.2560	0.3118
Expert vs `discard_only`	PASS	0.6643	+8.9716	[+8.7886, +9.1546]	45.2125	1.8971	0.2828
Expert vs `heuristic_balanced`	PASS	0.9961	+54.3673	[+54.0791, +54.6554]	64.9181	2.8003	0.2728
Expert vs `heuristic_cautious`	PASS	0.9972	+51.9397	[+51.6960, +52.1834]	75.2329	2.6090	0.2240

The new script bot is non-stalling in mirror play, symmetric under duplicate seat swap, and beats all three previous static bots by a clear positive score margin. Its own opened-color profile is in the intended 2-3 color band.

Part B - Ladder v2

Gate	Opponent	Result	Win rate (Wilson 95%)	Mean diff	CI95 diff	Length	Max-step	Opened colors	Play rate	Positive exp.
1	`discard_only`	PASS	1.0000 [0.9998, 1.0000]	+182.0670	[+181.6390, +182.4950]	82.1558	0.0000	4.2163	0.6398	3.1845
2	`heuristic_balanced`	PASS	0.9409 [0.9375, 0.9441]	+119.9250	[+118.9950, +120.8550]	151.2468	0.1859	4.9862	0.2937	3.5316
3	`heuristic_expert`	PASS	0.8382 [0.8330, 0.8432]	+43.3099	[+42.7065, +43.9133]	61.4025	0.0000	4.5509	0.6661	2.6121

Gate 1 passed the original win-rate and positive-expedition conditions. Gate 2 passed the mean-score-difference condition. Gate 3 passed the v2 condition: duplicate mean score difference is positive and its 95% confidence lower bound is above zero.

Canary Findings

The ladder did not produce the desired 2-3 color selectivity in the learned PPO policies. The gate-3 policy still opens 4.5509 colors/game against the expert. This is lower than the original gate-3 checkpoint but still far outside the target band. Treat the selectivity hypothesis as not confirmed.

Gate 2 also has a high forced-end canary: max_steps_rate=0.1859 in duplicate evaluation versus heuristic_balanced. It passed its score gate, but the balanced opponent still permits long games and 5-color farming.

Exploiter Baseline

After gate 3 passed, a new PPO exploiter was trained from random initialization against the frozen v2 gate-3 checkpoint with the same 250-update budget and duplicate-evaluated against that checkpoint.

Target checkpoint	Exploiter win rate (Wilson 95%)	Mean diff	CI95 diff	Length	Max-step	Opened colors	Play rate	Positive exp.
Old gate-3 checkpoint	0.6712 [0.6646, 0.6776]	+24.1371	n/a	48.0017	0.0000	4.8971	0.7461	2.1287
New v2 gate-3 checkpoint	0.8314 [0.8261, 0.8365]	+54.7721	[+53.9871, +55.5571]	60.8744	0.0001	4.9914	0.7028	2.8921

The new gate-3 checkpoint beats heuristic_expert, but is more exploitable under this protocol than the old checkpoint. That does not invalidate the gate, but it makes exploiter reduction a primary objective for the league phase.

Old vs New Gate-3 Canary Comparison

Metric	Old gate-3 vs cautious	New gate-3 vs expert	Interpretation
Mean score diff	+142.8993	+43.3099	New gate is harder and less suspiciously clean.
Win rate	0.9596	0.8382	New result is less inflated.
Opened colors	4.9896	4.5509	Improved, but still not selective enough.
Max-step rate	not recorded in old eval	0.0000	New expert evaluation does not stall.
Play action rate	0.2763	0.6661	New policy plays much more actively.
Exploiter win rate	0.6712	0.8314	New checkpoint is currently more exploitable.

Artifacts

Item	Path
Expert mirror JSON	`/mnt/2tbhdd/coolrl-lost-cities-artifacts/expert-bot/2026-07-05/expert-mirror-1000-duplicate.json`
Expert vs old bots JSONs	`/mnt/2tbhdd/coolrl-lost-cities-artifacts/expert-bot/2026-07-05/`
Gate 1 run	`/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_010429_jax-ppo-ladder-v2-discard-only/`
Gate 2 run	`/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_011827_jax-ppo-ladder-v2-balanced/`
Gate 3 run	`/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_013223_jax-ppo-ladder-v2-expert/`
Exploiter run	`/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_014713_jax-ppo-ladder-v2-exploiter/`

Decisions

The requested lost-cities-league-selfplay-spec.md file was not present in the repository. I created docs/plans/lost-cities-league-selfplay-spec.md as the tracked league spec location, following the repository docs routing for active plans.
Gate 2's high max-step rate is treated as a canary warning rather than a gate failure because the v2 instruction keeps gate 2's original score-based pass condition.

Proceed to league implementation only with the updated interpretation: anchors fix the Elo scale and provide style diversity, but they do not certify strength. The league's main success criterion should be lowering exploiter win rate from the v2 baseline of 0.83135 while preserving positive duplicate performance against heuristic_expert.

6.1 KiB Raw Blame History