coolguy/coorl-lost-cities
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Add diminishing returns diagnostic report

Browse Source
This commit is contained in:
coolguy
2026-07-05 18:32:47 +09:00
parent fa9a1c5286
commit 16cc31676c
6 changed files with 600 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reports/diminishing-returns-2026-07-05-expert.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

docs/reports/diminishing-returns-2026-07-05-exploiter.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 89 KiB

docs/reports/diminishing-returns-2026-07-05-h2h.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 57 KiB

docs/reports/diminishing-returns-2026-07-05-summary.jsonl
+17
View File
@@ -0,0 +1,17 @@
{"current": "league_v1_update_250", "current_label": "league v1 update 250", "event": "h2h_adjacent", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05/h2h_league_v1_update_250_vs_ladder_v2_gate3.json", "max_steps_rate": 0.0, "mean_game_length": 57.4935, "mean_score_diff": 42.87575, "opened_colors_per_game": 4.911, "play_action_rate": 0.6949448449578737, "positive_expeditions_per_game": 2.56825, "previous": "ladder_v2_gate3", "previous_label": "ladder v2 gate3", "score_diff_ci95_high": 44.43765342421665, "score_diff_ci95_low": 41.313846575783344, "score_diff_std": 50.40064352917072, "wilson_high": 0.8135690832007875, "wilson_low": 0.7888523724955226, "win_rate": 0.8015}
{"current": "league_v1_update_500", "current_label": "league v1 update 500", "event": "h2h_adjacent", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05/h2h_league_v1_update_500_vs_league_v1_update_250.json", "max_steps_rate": 0.0, "mean_game_length": 50.7755, "mean_score_diff": 5.20975, "opened_colors_per_game": 4.9415, "play_action_rate": 0.7485131356099098, "positive_expeditions_per_game": 2.351, "previous": "league_v1_update_250", "previous_label": "league v1 update 250", "score_diff_ci95_high": 6.664905283640598, "score_diff_ci95_low": 3.7545947163594007, "score_diff_std": 46.95601635366284, "wilson_high": 0.5558977420904201, "wilson_low": 0.525024543003114, "win_rate": 0.5405}
{"current": "repair_c01_update_500", "current_label": "repair c01 update 500", "event": "h2h_adjacent", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05/h2h_repair_c01_update_500_vs_league_v1_update_500.json", "max_steps_rate": 0.0, "mean_game_length": 49.2915, "mean_score_diff": 3.64125, "opened_colors_per_game": 4.9395, "play_action_rate": 0.7722021038110018, "positive_expeditions_per_game": 2.43225, "previous": "league_v1_update_500", "previous_label": "league v1 update 500", "score_diff_ci95_high": 5.0464015232057395, "score_diff_ci95_low": 2.23609847679426, "score_diff_std": 45.34245839244679, "wilson_high": 0.5476742803350019, "wilson_low": 0.5167638355728124, "win_rate": 0.53225}
{"checkpoint": "ladder_v2_gate3", "event": "expert_anchor", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05/expert_anchor_ladder_v2_gate3.json", "label": "ladder v2 gate3", "max_steps_rate": 0.0, "mean_game_length": 61.398, "mean_score_diff": 43.456, "opened_colors_per_game": 4.55925, "play_action_rate": 0.6680377805945128, "positive_expeditions_per_game": 2.61475, "score_diff_ci95_high": 44.81720694691559, "score_diff_ci95_low": 42.094793053084416, "score_diff_std": 43.92442262256479, "stage_index": 0, "wilson_high": 0.8490930348441887, "wilson_low": 0.8262583814919731, "win_rate": 0.838}
{"checkpoint": "league_v1_update_250", "event": "expert_anchor", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05/expert_anchor_league_v1_update_250.json", "label": "league v1 update 250", "max_steps_rate": 0.0, "mean_game_length": 56.52425, "mean_score_diff": 26.42325, "opened_colors_per_game": 4.54775, "play_action_rate": 0.6422064732439839, "positive_expeditions_per_game": 2.19725, "score_diff_ci95_high": 27.654455989944754, "score_diff_ci95_low": 25.192044010055245, "score_diff_std": 39.729456538778834, "stage_index": 1, "wilson_high": 0.7570405983993863, "wilson_low": 0.7299916729968844, "win_rate": 0.74375}
{"checkpoint": "league_v1_update_500", "event": "expert_anchor", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05/expert_anchor_league_v1_update_500.json", "label": "league v1 update 500", "max_steps_rate": 0.0, "mean_game_length": 53.29175, "mean_score_diff": 13.967, "opened_colors_per_game": 4.84725, "play_action_rate": 0.6934619506966774, "positive_expeditions_per_game": 2.14075, "score_diff_ci95_high": 15.054322005906956, "score_diff_ci95_low": 12.879677994093045, "score_diff_std": 35.086502770569304, "stage_index": 2, "wilson_high": 0.6609204517986065, "wilson_low": 0.6312989110391558, "win_rate": 0.64625}
{"checkpoint": "repair_c01_update_500", "event": "expert_anchor", "games": 4000, "json": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05/expert_anchor_repair_c01_update_500.json", "label": "repair c01 update 500", "max_steps_rate": 0.0, "mean_game_length": 54.61975, "mean_score_diff": 19.654, "opened_colors_per_game": 4.7015, "play_action_rate": 0.6708126417051431, "positive_expeditions_per_game": 2.1505, "score_diff_ci95_high": 20.81558482366198, "score_diff_ci95_low": 18.49241517633802, "score_diff_std": 37.482869759149246, "stage_index": 3, "wilson_high": 0.7068566468456556, "wilson_low": 0.6782734877661645, "win_rate": 0.69275}
{"budget": "same-budget phase-1 exploiter", "delta_from_previous_same_protocol": null, "event": "exploiter_trajectory", "mean_score_diff": 54.7721, "phase": "ladder_v2", "protocol": "short_random_shaping", "source": "docs/reports/ladder-v2-2026-07-05-summary.jsonl", "target": "ladder_v2_gate3", "warmstart": "random", "win_rate": 0.83135}
{"budget": "league v1 cycle exploiter", "delta_from_previous_same_protocol": null, "event": "exploiter_trajectory", "mean_score_diff": 0.6818, "phase": "league_v1", "protocol": "league_cycle_exploiter", "source": "docs/reports/league-v1-2026-07-05-summary.jsonl", "target": "league_v1_update_250", "warmstart": "random", "win_rate": 0.50225}
{"budget": "same exploiter vs final", "delta_from_previous_same_protocol": -0.031299999999999994, "event": "exploiter_trajectory", "mean_score_diff": -2.63775, "phase": "league_v1", "protocol": "league_cycle_exploiter", "source": "docs/reports/league-v1-2026-07-05-summary.jsonl", "target": "league_v1_update_500", "warmstart": "random", "win_rate": 0.47095}
{"budget": "1200 updates, shaping anneal", "delta_from_previous_same_protocol": null, "event": "exploiter_trajectory", "mean_score_diff": 10.95125, "phase": "gates_1_2_original", "protocol": "long_random_shaping", "source": "docs/reports/gates-1-2-2026-07-05-summary.jsonl", "target": "league_v1_update_500", "warmstart": "random", "win_rate": 0.587}
{"budget": "900 updates, shaping 0", "delta_from_previous_same_protocol": null, "event": "exploiter_trajectory", "mean_score_diff": 11.38075, "phase": "gates_1_2_original", "protocol": "warmstart_gate3_no_shaping", "source": "docs/reports/gates-1-2-2026-07-05-summary.jsonl", "target": "league_v1_update_500", "warmstart": "ladder_v2_gate3", "win_rate": 0.59}
{"budget": "900 updates, shaping 0", "delta_from_previous_same_protocol": null, "event": "exploiter_trajectory", "mean_score_diff": 10.42525, "phase": "gates_1_2_original", "protocol": "replay_exploiter_no_shaping", "source": "docs/reports/gates-1-2-2026-07-05-summary.jsonl", "target": "league_v1_update_500", "warmstart": "league_v1_cycle_1_exploiter", "win_rate": 0.582}
{"budget": "1200 updates, shaping anneal", "delta_from_previous_same_protocol": -0.04799999999999993, "event": "exploiter_trajectory", "mean_score_diff": 6.27925, "phase": "repair_c01", "protocol": "long_random_shaping", "source": "docs/reports/gates-1-2-2026-07-05-summary.jsonl", "target": "repair_c01_update_500", "warmstart": "random", "win_rate": 0.539}
{"budget": "900 updates, shaping 0", "delta_from_previous_same_protocol": -0.040999999999999925, "event": "exploiter_trajectory", "mean_score_diff": 7.0285, "phase": "repair_c01", "protocol": "warmstart_gate3_no_shaping", "source": "docs/reports/gates-1-2-2026-07-05-summary.jsonl", "target": "repair_c01_update_500", "warmstart": "ladder_v2_gate3", "win_rate": 0.549}
{"budget": "900 updates, shaping 0", "delta_from_previous_same_protocol": -0.042749999999999955, "event": "exploiter_trajectory", "mean_score_diff": 6.10225, "phase": "repair_c01", "protocol": "replay_exploiter_no_shaping", "source": "docs/reports/gates-1-2-2026-07-05-summary.jsonl", "target": "repair_c01_update_500", "warmstart": "league_v1_cycle_1_exploiter", "win_rate": 0.53925}
{"artifact_dir": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05", "elapsed_seconds": 15.667906627990305, "event": "diminishing_returns_complete", "report": "docs/reports/diminishing-returns-2026-07-05.md"}
docs/reports/diminishing-returns-2026-07-05.md
+65
View File
@@ -0,0 +1,65 @@
# Diminishing Returns Diagnostic - 2026-07-05
신규 학습 없이 기존 체크포인트와 평가 롤아웃만 사용했다. 모든 duplicate 평가는 셔플 뱅크 seed 20260704, 2,000쌍(4000 games) 기준이다.
Raw artifacts: `/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05`
## Checkpoints
| Order | Name | Config | Checkpoint |
| ---: | --- | --- | --- |
| 0 | `ladder_v2_gate3` | `configs/jax_ppo/ladder-v2-expert.yaml` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_013223_jax-ppo-ladder-v2-expert/latest` |
| 1 | `league_v1_update_250` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/main_ppo_config.json` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/snapshots/cycle_01_update_000250` |
| 2 | `league_v1_update_500` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/main_ppo_config.json` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/snapshots/cycle_01_update_000500` |
| 3 | `repair_c01_update_500` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/gates-1-2/2026-07-05_094238_gates-1-2/gate2c/league/2026-07-05_125709_jax-ppo-gates-1-2-repair-c01/main_ppo_config.json` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/gates-1-2/2026-07-05_094238_gates-1-2/gate2c/league/2026-07-05_125709_jax-ppo-gates-1-2-repair-c01/snapshots/cycle_01_update_000500` |
## 1. Adjacent Head-to-Head
![Adjacent H2H](diminishing-returns-2026-07-05-h2h.png)
| Later | Earlier | Win rate | Wilson CI | Mean diff | Score CI | Opened colors | Max-step |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| `league_v1_update_250` | `ladder_v2_gate3` | 0.8015 | [0.7889, 0.8136] | +42.8757 | [+41.3138, +44.4377] | 4.9110 | 0.0000 |
| `league_v1_update_500` | `league_v1_update_250` | 0.5405 | [0.5250, 0.5559] | +5.2097 | [+3.7546, +6.6649] | 4.9415 | 0.0000 |
| `repair_c01_update_500` | `league_v1_update_500` | 0.5323 | [0.5168, 0.5477] | +3.6412 | [+2.2361, +5.0464] | 4.9395 | 0.0000 |
## 2. External Anchor Trajectory
![Expert anchor](diminishing-returns-2026-07-05-expert.png)
| Checkpoint | Win rate | Wilson CI | Mean diff | Score CI | Opened colors | Max-step |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| `ladder_v2_gate3` | 0.8380 | [0.8263, 0.8491] | +43.4560 | [+42.0948, +44.8172] | 4.5592 | 0.0000 |
| `league_v1_update_250` | 0.7438 | [0.7300, 0.7570] | +26.4232 | [+25.1920, +27.6545] | 4.5477 | 0.0000 |
| `league_v1_update_500` | 0.6462 | [0.6313, 0.6609] | +13.9670 | [+12.8797, +15.0543] | 4.8472 | 0.0000 |
| `repair_c01_update_500` | 0.6927 | [0.6783, 0.7069] | +19.6540 | [+18.4924, +20.8156] | 4.7015 | 0.0000 |
Recent expert-anchor slope: `+5.6870` points (league_v1_update_500 -> repair_c01_update_500).
## 3. Exploitability Trajectory
![Exploitability](diminishing-returns-2026-07-05-exploiter.png)
프로토콜이 다른 피탈률은 한 곡선에 섞지 않았다. `Delta`는 같은 프로토콜 안에서만 계산했다.
| Phase | Target | Protocol | Budget | Warm start | Win rate | Delta same protocol | Mean diff | Source |
| --- | --- | --- | --- | --- | ---: | ---: | ---: | --- |
| `ladder_v2` | `ladder_v2_gate3` | `short_random_shaping` | same-budget phase-1 exploiter | `random` | 0.83135 | n/a | +54.7721 | [ladder-v2-2026-07-05-summary.jsonl](ladder-v2-2026-07-05-summary.jsonl) |
| `league_v1` | `league_v1_update_250` | `league_cycle_exploiter` | league v1 cycle exploiter | `random` | 0.50225 | n/a | +0.6818 | [league-v1-2026-07-05-summary.jsonl](league-v1-2026-07-05-summary.jsonl) |
| `league_v1` | `league_v1_update_500` | `league_cycle_exploiter` | same exploiter vs final | `random` | 0.47095 | -0.03130 | -2.6378 | [league-v1-2026-07-05-summary.jsonl](league-v1-2026-07-05-summary.jsonl) |
| `gates_1_2_original` | `league_v1_update_500` | `long_random_shaping` | 1200 updates, shaping anneal | `random` | 0.58700 | n/a | +10.9512 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `gates_1_2_original` | `league_v1_update_500` | `warmstart_gate3_no_shaping` | 900 updates, shaping 0 | `ladder_v2_gate3` | 0.59000 | n/a | +11.3808 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `gates_1_2_original` | `league_v1_update_500` | `replay_exploiter_no_shaping` | 900 updates, shaping 0 | `league_v1_cycle_1_exploiter` | 0.58200 | n/a | +10.4253 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `repair_c01` | `repair_c01_update_500` | `long_random_shaping` | 1200 updates, shaping anneal | `random` | 0.53900 | -0.04800 | +6.2793 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `repair_c01` | `repair_c01_update_500` | `warmstart_gate3_no_shaping` | 900 updates, shaping 0 | `ladder_v2_gate3` | 0.54900 | -0.04100 | +7.0285 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `repair_c01` | `repair_c01_update_500` | `replay_exploiter_no_shaping` | 900 updates, shaping 0 | `league_v1_cycle_1_exploiter` | 0.53925 | -0.04275 | +6.1022 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
## Judgment
추가 학습 1사이클(~1.5시간 GPU)의 기대 개선은 인접 H2H 최근 이득 +3.64점, expert 앵커 최근 변화 +5.69점, 동일 프로토콜 exploiter 평균 피탈률 감소 0.044에 따라 대략 +3.6~+5.7점 또는 피탈률 -0.044 내외 수준으로 추정되며, 수확체감 구간에 진입했다. 근거는 최근 H2H 이득이 첫 리그 전이보다 작고, expert 앵커 성능이 v1 update 250 이후 하락/회복을 반복하며, 보수 후 exploiter 최악값이 0.549로 게이트 바로 아래에 머문다는 세 측정의 일치다. 단, repair가 expert 앵커를 유의하게 회복시킨 점은 별도 긍정 신호다.
## Notes
- H2H와 expert 앵커 평가는 이번 작업에서 새로 실행했다.
- Exploiter 표는 지금까지 생성된 리포트/JSONL의 기존 측정값만 재정리했다.
- 이 작업에서는 신규 학습, 봇 수정, 체크포인트 수정이 없었다.