69 lines
3.0 KiB
Markdown
69 lines
3.0 KiB
Markdown
# JAX PPO Static-Opponent Ladder Report - 2026-07-04
|
|
|
|
**Status:** PASS.
|
|
**Implementation commit:** `4c0c2e9`.
|
|
**Hardware:** RTX 3090 with optional CUDA JAX via `uv run --with 'jax[cuda12]'`.
|
|
**Artifact root:** `/mnt/2tbhdd/coolrl-lost-cities-artifacts/jax-ppo-static-opponents/`.
|
|
|
|
## Protocol
|
|
|
|
Each opponent was trained from scratch with the same PPO configuration:
|
|
|
|
- `batch_games=8192`
|
|
- `rollout_steps=400`
|
|
- `total_updates=250`
|
|
- `gamma=1.0`
|
|
- `gae_lambda=0.95`
|
|
- `clip_epsilon=0.2`
|
|
- `entropy_coef=0.01`
|
|
- `seed=20260704`
|
|
|
|
Evaluation used a fixed 10,000-deck shuffle bank and duplicate play: each deck
|
|
was played once with the learner in seat 0 and once with the learner in seat 1,
|
|
for 20,000 evaluated games per gate. Win rates include Wilson 95% intervals.
|
|
|
|
## Results
|
|
|
|
| Gate | Opponent | Result | Win rate (Wilson 95%) | Mean score diff | Mean game length | Positive expeditions/game |
|
|
| --- | --- | --- | --- | ---: | ---: | ---: |
|
|
| 1 | `discard_only` | PASS | 1.00000 [0.99981, 1.00000] | 204.56335 | 82.45495 | 3.2783 |
|
|
| 2 | `heuristic_balanced` | PASS | 0.98655 [0.98486, 0.98806] | 116.83400 | 167.81450 | 3.9573 |
|
|
| 3 | `heuristic_cautious` | PASS | 0.95955 [0.95673, 0.96219] | 142.89930 | 185.38690 | 4.0946 |
|
|
|
|
Gate 1 passed both required conditions: win rate >= 90% and at least two
|
|
positive expeditions per game. Gates 2 and 3 passed their mean-score-difference
|
|
condition.
|
|
|
|
## Artifacts
|
|
|
|
| Opponent | Run directory | Evaluation JSON |
|
|
| --- | --- | --- |
|
|
| `discard_only` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/jax-ppo-static-opponents/2026-07-04_223401_jax-ppo-discard-only/` | `eval_discard_only_duplicate.json` |
|
|
| `heuristic_balanced` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/jax-ppo-static-opponents/2026-07-04_224749_jax-ppo-balanced/` | `eval_balanced_duplicate.json` |
|
|
| `heuristic_cautious` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/jax-ppo-static-opponents/2026-07-04_230150_jax-ppo-cautious/` | `eval_cautious_duplicate.json` |
|
|
|
|
The random rollout baseline against `discard_only` is stored at:
|
|
|
|
```text
|
|
/mnt/2tbhdd/coolrl-lost-cities-artifacts/jax-ppo-static-opponents/random_baseline_discard_only.json
|
|
```
|
|
|
|
## Canary Notes
|
|
|
|
The random-policy baseline against `discard_only` had `play_action_rate=0.28855`
|
|
and only `0.41797` positive expeditions per game. The trained discard-only
|
|
policy ended duplicate evaluation at `play_action_rate=0.64598` and `3.2783`
|
|
positive expeditions per game, so the intended anti-collapse signal is present.
|
|
|
|
Balanced and cautious checkpoints pass their score gates decisively, but their
|
|
duplicate evaluations are longer: mean game lengths are `167.81450` and
|
|
`185.38690`. The final training rollout forced-end rates were low but nonzero
|
|
for those opponents (`0.00415` and `0.00391`). Treat game length and forced-end
|
|
rate as canaries in the next self-play phase.
|
|
|
|
## Next Step
|
|
|
|
The static ladder has cleared. The next phase can start snapshot-pool league
|
|
self-play, using these three passed checkpoints as initial anchors and keeping
|
|
duplicate evaluation as the regression gate.
|