# JAX PPO Ladder v2 Report - 2026-07-05

**Status:** PASS for the ordered ladder gates; canary warnings remain.
**Hardware:** RTX 3090 via `uv run --with 'jax[cuda12]'`.
**Artifact roots:**

- Expert bot: `/mnt/2tbhdd/coolrl-lost-cities-artifacts/expert-bot/2026-07-05/`
- Ladder v2: `/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/`

## Protocol

Part A added a pure-JAX `heuristic_expert` opponent and evaluated it with a
fixed shuffle bank plus duplicate play. Part B trained PPO from random
initialization for each gate with the same 250-update configuration used by the
original ladder, changing only the opponent and artifact root:

- Gate 1: `discard_only`
- Gate 2: `heuristic_balanced`
- Gate 3: `heuristic_expert`

All evaluations below used 10,000 deck orders with duplicate seat-swapped play,
for 20,000 evaluated games.

## Part A - Expert Bot Gate

| Check | Result | Win rate | Mean diff | CI95 diff | Length | Max-step | Opened colors | Play rate |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Expert mirror, 1,000 duplicate pairs | PASS | 0.4815 | 0.0000 | [-0.6956, +0.6956] | 45.1560 | 0.0000 | 2.2560 | 0.3118 |
| Expert vs `discard_only` | PASS | 0.6643 | +8.9716 | [+8.7886, +9.1546] | 45.2125 | 0.0000 | 1.8971 | 0.2828 |
| Expert vs `heuristic_balanced` | PASS | 0.9961 | +54.3673 | [+54.0791, +54.6554] | 64.9181 | 0.0000 | 2.8003 | 0.2728 |
| Expert vs `heuristic_cautious` | PASS | 0.9972 | +51.9397 | [+51.6960, +52.1834] | 75.2329 | 0.0000 | 2.6090 | 0.2240 |

The new script bot is non-stalling in mirror play, symmetric under duplicate
seat swap, and beats all three previous static bots by a clear positive score
margin. Its own opened-color profile is in the intended 2-3 color band.

## Part B - Ladder v2

| Gate | Opponent | Result | Win rate (Wilson 95%) | Mean diff | CI95 diff | Length | Max-step | Opened colors | Play rate | Positive exp. |
| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | `discard_only` | PASS | 1.0000 [0.9998, 1.0000] | +182.0670 | [+181.6390, +182.4950] | 82.1558 | 0.0000 | 4.2163 | 0.6398 | 3.1845 |
| 2 | `heuristic_balanced` | PASS | 0.9409 [0.9375, 0.9441] | +119.9250 | [+118.9950, +120.8550] | 151.2468 | 0.1859 | 4.9862 | 0.2937 | 3.5316 |
| 3 | `heuristic_expert` | PASS | 0.8382 [0.8330, 0.8432] | +43.3099 | [+42.7065, +43.9133] | 61.4025 | 0.0000 | 4.5509 | 0.6661 | 2.6121 |

Gate 1 passed the original win-rate and positive-expedition conditions. Gate 2
passed the mean-score-difference condition. Gate 3 passed the v2 condition:
duplicate mean score difference is positive and its 95% confidence lower bound
is above zero.

## Canary Findings

The ladder did not produce the desired 2-3 color selectivity in the learned PPO
policies. The gate-3 policy still opens 4.5509 colors/game against the expert.
This is lower than the original gate-3 checkpoint but still far outside the
target band. Treat the selectivity hypothesis as not confirmed.

Gate 2 also has a high forced-end canary: `max_steps_rate=0.1859` in duplicate
evaluation versus `heuristic_balanced`. It passed its score gate, but the
balanced opponent still permits long games and 5-color farming.

## Exploiter Baseline

After gate 3 passed, a new PPO exploiter was trained from random initialization
against the frozen v2 gate-3 checkpoint with the same 250-update budget and
duplicate-evaluated against that checkpoint.

| Target checkpoint | Exploiter win rate (Wilson 95%) | Mean diff | CI95 diff | Length | Max-step | Opened colors | Play rate | Positive exp. |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Old gate-3 checkpoint | 0.6712 [0.6646, 0.6776] | +24.1371 | n/a | 48.0017 | 0.0000 | 4.8971 | 0.7461 | 2.1287 |
| New v2 gate-3 checkpoint | 0.8314 [0.8261, 0.8365] | +54.7721 | [+53.9871, +55.5571] | 60.8744 | 0.0001 | 4.9914 | 0.7028 | 2.8921 |

The new gate-3 checkpoint beats `heuristic_expert`, but is more exploitable
under this protocol than the old checkpoint. That does not invalidate the gate,
but it makes exploiter reduction a primary objective for the league phase.

## Old vs New Gate-3 Canary Comparison

| Metric | Old gate-3 vs cautious | New gate-3 vs expert | Interpretation |
| --- | ---: | ---: | --- |
| Mean score diff | +142.8993 | +43.3099 | New gate is harder and less suspiciously clean. |
| Win rate | 0.9596 | 0.8382 | New result is less inflated. |
| Opened colors | 4.9896 | 4.5509 | Improved, but still not selective enough. |
| Max-step rate | not recorded in old eval | 0.0000 | New expert evaluation does not stall. |
| Play action rate | 0.2763 | 0.6661 | New policy plays much more actively. |
| Exploiter win rate | 0.6712 | 0.8314 | New checkpoint is currently more exploitable. |

## Artifacts

| Item | Path |
| --- | --- |
| Expert mirror JSON | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/expert-bot/2026-07-05/expert-mirror-1000-duplicate.json` |
| Expert vs old bots JSONs | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/expert-bot/2026-07-05/` |
| Gate 1 run | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_010429_jax-ppo-ladder-v2-discard-only/` |
| Gate 2 run | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_011827_jax-ppo-ladder-v2-balanced/` |
| Gate 3 run | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_013223_jax-ppo-ladder-v2-expert/` |
| Exploiter run | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_014713_jax-ppo-ladder-v2-exploiter/` |

## Decisions

- The requested `lost-cities-league-selfplay-spec.md` file was not present in
  the repository. I created `docs/plans/lost-cities-league-selfplay-spec.md` as
  the tracked league spec location, following the repository docs routing for
  active plans.
- Gate 2's high max-step rate is treated as a canary warning rather than a
  gate failure because the v2 instruction keeps gate 2's original score-based
  pass condition.

## Next

Proceed to league implementation only with the updated interpretation: anchors
fix the Elo scale and provide style diversity, but they do not certify strength.
The league's main success criterion should be lowering exploiter win rate from
the v2 baseline of `0.83135` while preserving positive duplicate performance
against `heuristic_expert`.
