# Diminishing Returns Diagnostic - 2026-07-05

신규 학습 없이 기존 체크포인트와 평가 롤아웃만 사용했다. 모든 duplicate 평가는 셔플 뱅크 seed 20260704, 2,000쌍(4000 games) 기준이다.

Raw artifacts: `/mnt/2tbhdd/coolrl-lost-cities-artifacts/diminishing-returns/2026-07-05`

## Checkpoints

| Order | Name | Config | Checkpoint |
| ---: | --- | --- | --- |
| 0 | `ladder_v2_gate3` | `configs/jax_ppo/ladder-v2-expert.yaml` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/ladder-v2/2026-07-05_013223_jax-ppo-ladder-v2-expert/latest` |
| 1 | `league_v1_update_250` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/main_ppo_config.json` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/snapshots/cycle_01_update_000250` |
| 2 | `league_v1_update_500` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/main_ppo_config.json` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/snapshots/cycle_01_update_000500` |
| 3 | `repair_c01_update_500` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/gates-1-2/2026-07-05_094238_gates-1-2/gate2c/league/2026-07-05_125709_jax-ppo-gates-1-2-repair-c01/main_ppo_config.json` | `/mnt/2tbhdd/coolrl-lost-cities-artifacts/gates-1-2/2026-07-05_094238_gates-1-2/gate2c/league/2026-07-05_125709_jax-ppo-gates-1-2-repair-c01/snapshots/cycle_01_update_000500` |

## 1. Adjacent Head-to-Head

![Adjacent H2H](diminishing-returns-2026-07-05-h2h.png)

| Later | Earlier | Win rate | Wilson CI | Mean diff | Score CI | Opened colors | Max-step |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| `league_v1_update_250` | `ladder_v2_gate3` | 0.8015 | [0.7889, 0.8136] | +42.8757 | [+41.3138, +44.4377] | 4.9110 | 0.0000 |
| `league_v1_update_500` | `league_v1_update_250` | 0.5405 | [0.5250, 0.5559] | +5.2097 | [+3.7546, +6.6649] | 4.9415 | 0.0000 |
| `repair_c01_update_500` | `league_v1_update_500` | 0.5323 | [0.5168, 0.5477] | +3.6412 | [+2.2361, +5.0464] | 4.9395 | 0.0000 |

## 2. External Anchor Trajectory

![Expert anchor](diminishing-returns-2026-07-05-expert.png)

| Checkpoint | Win rate | Wilson CI | Mean diff | Score CI | Opened colors | Max-step |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| `ladder_v2_gate3` | 0.8380 | [0.8263, 0.8491] | +43.4560 | [+42.0948, +44.8172] | 4.5592 | 0.0000 |
| `league_v1_update_250` | 0.7438 | [0.7300, 0.7570] | +26.4232 | [+25.1920, +27.6545] | 4.5477 | 0.0000 |
| `league_v1_update_500` | 0.6462 | [0.6313, 0.6609] | +13.9670 | [+12.8797, +15.0543] | 4.8472 | 0.0000 |
| `repair_c01_update_500` | 0.6927 | [0.6783, 0.7069] | +19.6540 | [+18.4924, +20.8156] | 4.7015 | 0.0000 |

Recent expert-anchor slope: `+5.6870` points (league_v1_update_500 -> repair_c01_update_500).

## 3. Exploitability Trajectory

![Exploitability](diminishing-returns-2026-07-05-exploiter.png)

프로토콜이 다른 피탈률은 한 곡선에 섞지 않았다. `Delta`는 같은 프로토콜 안에서만 계산했다.

| Phase | Target | Protocol | Budget | Warm start | Win rate | Delta same protocol | Mean diff | Source |
| --- | --- | --- | --- | --- | ---: | ---: | ---: | --- |
| `ladder_v2` | `ladder_v2_gate3` | `short_random_shaping` | same-budget phase-1 exploiter | `random` | 0.83135 | n/a | +54.7721 | [ladder-v2-2026-07-05-summary.jsonl](ladder-v2-2026-07-05-summary.jsonl) |
| `league_v1` | `league_v1_update_250` | `league_cycle_exploiter` | league v1 cycle exploiter | `random` | 0.50225 | n/a | +0.6818 | [league-v1-2026-07-05-summary.jsonl](league-v1-2026-07-05-summary.jsonl) |
| `league_v1` | `league_v1_update_500` | `league_cycle_exploiter` | same exploiter vs final | `random` | 0.47095 | -0.03130 | -2.6378 | [league-v1-2026-07-05-summary.jsonl](league-v1-2026-07-05-summary.jsonl) |
| `gates_1_2_original` | `league_v1_update_500` | `long_random_shaping` | 1200 updates, shaping anneal | `random` | 0.58700 | n/a | +10.9512 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `gates_1_2_original` | `league_v1_update_500` | `warmstart_gate3_no_shaping` | 900 updates, shaping 0 | `ladder_v2_gate3` | 0.59000 | n/a | +11.3808 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `gates_1_2_original` | `league_v1_update_500` | `replay_exploiter_no_shaping` | 900 updates, shaping 0 | `league_v1_cycle_1_exploiter` | 0.58200 | n/a | +10.4253 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `repair_c01` | `repair_c01_update_500` | `long_random_shaping` | 1200 updates, shaping anneal | `random` | 0.53900 | -0.04800 | +6.2793 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `repair_c01` | `repair_c01_update_500` | `warmstart_gate3_no_shaping` | 900 updates, shaping 0 | `ladder_v2_gate3` | 0.54900 | -0.04100 | +7.0285 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |
| `repair_c01` | `repair_c01_update_500` | `replay_exploiter_no_shaping` | 900 updates, shaping 0 | `league_v1_cycle_1_exploiter` | 0.53925 | -0.04275 | +6.1022 | [gates-1-2-2026-07-05-summary.jsonl](gates-1-2-2026-07-05-summary.jsonl) |

## Judgment

추가 학습 1사이클(~1.5시간 GPU)의 기대 개선은 인접 H2H 최근 이득 +3.64점, expert 앵커 최근 변화 +5.69점, 동일 프로토콜 exploiter 평균 피탈률 감소 0.044에 따라 대략 +3.6~+5.7점 또는 피탈률 -0.044 내외 수준으로 추정되며, 수확체감 구간에 진입했다. 근거는 최근 H2H 이득이 첫 리그 전이보다 작고, expert 앵커 성능이 v1 update 250 이후 하락/회복을 반복하며, 보수 후 exploiter 최악값이 0.549로 게이트 바로 아래에 머문다는 세 측정의 일치다. 단, repair가 expert 앵커를 유의하게 회복시킨 점은 별도 긍정 신호다.

## Notes

- H2H와 expert 앵커 평가는 이번 작업에서 새로 실행했다.
- Exploiter 표는 지금까지 생성된 리포트/JSONL의 기존 측정값만 재정리했다.
- 이 작업에서는 신규 학습, 봇 수정, 체크포인트 수정이 없었다.
