coolguy/coorl-lost-cities
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Add final cycle report and human play CLI

Browse Source
This commit is contained in:
coolguy
2026-07-06 00:14:24 +09:00
parent 16cc31676c
commit 19560029b5
6 changed files with 1398 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reports/final-cycles-and-human-play-2026-07-05-summary.jsonl
+8
View File
@@ -0,0 +1,8 @@
{"cycle": 1, "event": "final_cycle_league", "league_config": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/league_config.yaml", "league_run_dir": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01", "target_checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/snapshots/cycle_01_update_000500", "target_config": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/main_ppo_config.json", "worst_protocol_used": "warmstart_gate3"}
{"checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_205759_final-c01-long-random-exploiter/latest", "cycle": 1, "event": "final_cycle_exploiter", "exploiter": "long_random", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_205759_final-c01-long-random-exploiter/eval_vs_final_cycle_01_duplicate.json", "losses": 919.0, "max_steps_rate": 0.0005, "mean_game_length": 65.69525, "mean_score_diff": 55.799, "notes": "random init + shaping anneal, 1200 updates", "opened_colors_per_game": 4.92425, "play_action_rate": 0.5457588946038896, "positive_expeditions_per_game": 2.7735, "resume": null, "score_diff_ci95_high": 57.963152016622004, "score_diff_ci95_low": 53.634847983377995, "target": "final_cycle_01", "target_checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/snapshots/cycle_01_update_000500", "target_config": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/main_ppo_config.json", "ties": 22.0, "train_dir": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_205759_final-c01-long-random-exploiter", "wilson_high": 0.7776365926294233, "wilson_low": 0.7513553821486798, "win_rate": 0.76475, "wins": 3059.0}
{"checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_221402_final-c01-warmstart-gate3-exploiter/latest", "cycle": 1, "event": "final_cycle_exploiter", "exploiter": "warmstart_gate3", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_221402_final-c01-warmstart-gate3-exploiter/eval_vs_final_cycle_01_duplicate.json", "losses": 961.0, "max_steps_rate": 0.00075, "mean_game_length": 65.13875, "mean_score_diff": 53.30925, "notes": "ladder v2 gate-3 warm start, shaping disabled, 900 updates", "opened_colors_per_game": 4.90075, "play_action_rate": 0.5480372138722999, "positive_expeditions_per_game": 2.71225, "resume": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_013223_jax-ppo-ladder-v2-expert/latest", "score_diff_ci95_high": 55.52225515395847, "score_diff_ci95_low": 51.09624484604153, "target": "final_cycle_01", "target_checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/snapshots/cycle_01_update_000500", "target_config": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/main_ppo_config.json", "ties": 18.0, "train_dir": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_221402_final-c01-warmstart-gate3-exploiter", "wilson_high": 0.7683246594694194, "wilson_low": 0.741685544731188, "win_rate": 0.75525, "wins": 3021.0}
{"checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_231108_final-c01-replay-exploiter-exploiter/latest", "cycle": 1, "event": "final_cycle_exploiter", "exploiter": "replay_exploiter", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_231108_final-c01-replay-exploiter-exploiter/eval_vs_final_cycle_01_duplicate.json", "losses": 866.0, "max_steps_rate": 0.00025, "mean_game_length": 65.467, "mean_score_diff": 59.351, "notes": "league v1 cycle-1 exploiter warm start, shaping disabled, 900 updates", "opened_colors_per_game": 4.989, "play_action_rate": 0.5882132820136853, "positive_expeditions_per_game": 2.7905, "resume": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/exploiters/2026-07-05_060727_jax-ppo-league-v1-cycle-1-exploiter/latest", "score_diff_ci95_high": 61.580508518304526, "score_diff_ci95_low": 57.12149148169547, "target": "final_cycle_01", "target_checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/snapshots/cycle_01_update_000500", "target_config": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/main_ppo_config.json", "ties": 17.0, "train_dir": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/battery/2026-07-05_231108_final-c01-replay-exploiter-exploiter", "wilson_high": 0.7918317711217939, "wilson_low": 0.7661323798009079, "win_rate": 0.77925, "wins": 3117.0}
{"cycle": 1, "event": "final_cycle_battery_judgment", "passed": false, "threshold": 0.52, "worst_exploiter": "replay_exploiter", "worst_win_rate": 0.77925}
{"cycle": 1, "event": "final_cycle_guard_expert", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/guard_vs_heuristic_expert.json", "losses": 852.0, "max_steps_rate": 0.0, "mean_game_length": 60.38125, "mean_score_diff": 36.08475, "opened_colors_per_game": 4.15225, "passed": true, "play_action_rate": 0.5845773004045489, "positive_expeditions_per_game": 2.05975, "score_diff_ci95_high": 37.5625235448534, "score_diff_ci95_low": 34.6069764551466, "target": "final_cycle_01", "target_checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/snapshots/cycle_01_update_000500", "target_config": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/main_ppo_config.json", "ties": 33.0, "wilson_high": 0.7913426526326875, "wilson_low": 0.7656224577333041, "win_rate": 0.77875, "wins": 3115.0}
{"current": "final_cycle_01", "current_checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/snapshots/cycle_01_update_000500", "cycle": 1, "event": "final_cycle_h2h", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/cycle_01/h2h_final_cycle_01_vs_repair_c01_update_500.json", "losses": 1987.0, "max_steps_rate": 0.0, "mean_game_length": 49.54275, "mean_score_diff": 0.35425, "opened_colors_per_game": 4.91275, "play_action_rate": 0.7646239835347768, "positive_expeditions_per_game": 2.422, "previous": "repair_c01_update_500", "previous_checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/gates-1-2/2026-07-05_094238_gates-1-2/gate2c/league/2026-07-05_125709_jax-ppo-gates-1-2-repair-c01/snapshots/cycle_01_update_000500", "score_diff_ci95_high": 1.7891388254913125, "score_diff_ci95_low": -1.0806388254913126, "ties": 51.0, "wilson_high": 0.505993762454951, "wilson_low": 0.47502446696755335, "win_rate": 0.4905, "wins": 1962.0}
{"elapsed_seconds": 17320.886204754002, "event": "final_cycles_complete", "final_candidate": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/final_candidate", "final_config": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/main_ppo_config.json", "source_checkpoint": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/snapshots/cycle_01_update_000500", "stop_reason": "h2h_stagnation"}
docs/reports/final-cycles-and-human-play-2026-07-05.md
+83
View File
@@ -0,0 +1,83 @@
# Final Cycles and Human Play - 2026-07-05
## Part A - Closing Reinforcement Cycles
Raw artifacts: `/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05`
Stop reason: `h2h_stagnation`
Final candidate: `/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/final_candidate`
Final candidate config: `/mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/league/2026-07-05_191933_jax-ppo-final-cycle-c01/main_ppo_config.json`
The initial 4 hour GPU budget was treated as an estimate after user confirmation,
not a hard cutoff. The run completed cycle 1 fully, then stopped by the written
H2H stagnation rule. Cycle 2 was not started.
Bottom line: the closing reinforcement cycle did not produce a materially better
policy. The cycle-1 snapshot passed the expert guard, but the strengthened
exploiter battery got substantially worse than the incoming baseline target:
worst exploiter win rate was 0.7792, far above the 0.5200 success threshold. The
adjacent H2H against `repair_c01_update_500` was statistically indistinguishable
from zero, so this recipe is stopped here. Per instruction, this report does not
recommend more training with the same observation/network/PPO recipe.
### Expert Guard
| Cycle | Passed | Win rate | Mean diff | CI low | Opened colors | Max-step |
| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 1 | True | 0.7788 | +36.0847 | +34.6070 | 4.1523 | 0.0000 |
### Adjacent H2H
| Cycle | Current | Previous | Win rate | Mean diff | Score CI |
| ---: | --- | --- | ---: | ---: | ---: |
| 1 | `final_cycle_01` | `repair_c01_update_500` | 0.4905 | +0.3543 | [-1.0806, +1.7891] |
### Strengthened Exploiter Battery
| Cycle | Exploiter | Win rate | Mean diff | CI low | Opened colors | Max-step |
| ---: | --- | ---: | ---: | ---: | ---: | ---: |
| 1 | `long_random` | 0.7648 | +55.7990 | +53.6348 | 4.9242 | 0.0005 |
| 1 | `warmstart_gate3` | 0.7552 | +53.3092 | +51.0962 | 4.9008 | 0.0008 |
| 1 | `replay_exploiter` | 0.7792 | +59.3510 | +57.1215 | 4.9890 | 0.0003 |
### Battery Judgment
| Cycle | Worst exploiter | Worst win rate | Threshold | Passed |
| ---: | --- | ---: | ---: | ---: |
| 1 | `replay_exploiter` | 0.7792 | 0.5200 | False |
## Part B - Human Play Interface
Start a single game:
```bash
uv run --with 'jax[cuda12]' lost-cities-jax-ppo play \
--checkpoint /mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/final_candidate \
--seat 0
```
Start a duplicate set with one shared shuffle and swapped seats:
```bash
uv run --with 'jax[cuda12]' lost-cities-jax-ppo play \
--checkpoint /mnt/2tbhdd/coolrl-lost-cities-artifacts/final-cycles/2026-07-05/final_candidate \
--seat 0 --duplicate
```
Summarize logged human games:
```bash
uv run lost-cities-jax-ppo human-play summarize \
--log-dir /mnt/2tbhdd/coolrl-lost-cities-artifacts/human-play/
```
Move syntax: `play R7 draw deck`, `discard G3 draw Y`, or `play RHS draw deck`.
The renderer shows only the human hand, both boards, all public discard piles, deck count, and current board score differential. Opponent hand and deck order are not rendered.
Every game is appended to `/mnt/2tbhdd/coolrl-lost-cities-artifacts/human-play/games.jsonl` with deck seed/index, full action list, AI top-3 policy actions/probabilities, value outputs, scoring breakdown, and optional human comment.
Validation performed before final report:
- `uv run ruff check .` passed.
- `uv run pytest -q` passed: 257 passed, 1 skipped.
- `uv run lost-cities-jax-ppo play --help` passed.
- `uv run lost-cities-jax-ppo human-play summarize --log-dir /tmp/nonexistent-human-play-log` passed.
- `scripts/librarian.sh` found no link or code-citation errors; it still exits non-zero on the known pre-existing `docs/plans/deep-cfr-selectivity.md` 500-line soft cap.