From cc12b058e541c2e662821229acced5c65c9d2c04 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=A0=95=EC=8B=9C=EC=9B=90?= Date: Wed, 15 Jul 2026 05:26:38 +0900 Subject: [PATCH] Re-measure both-seat vs privileged critic at the same budget The 39.3M ablation was the critic's worst ground. At a matched 131M budget: both-seat training is worth +17.1 points of win rate, the privileged critic +3.7 -- a 4.6x gap. And they behave differently: both-seat is flat across scale (+16.8 at 39.3M, +17.1 at 131M) while the critic swings (-1.6 to +3.7). The mechanism explains it. Both-seat doubles the data at the same compute -- self-play already played those plies with the same network. The critic adds no data, only accuracy, and a privileged value trunk has more to fit, so it grades badly until there is enough to fit it with. Fable ranked the critic first among missing ideas and both-seat fourth. That was exactly backwards. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh --- docs/plans/lost-cities-classic-3round.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/plans/lost-cities-classic-3round.md b/docs/plans/lost-cities-classic-3round.md index 7736435..ea66508 100644 --- a/docs/plans/lost-cities-classic-3round.md +++ b/docs/plans/lost-cities-classic-3round.md @@ -515,3 +515,27 @@ Fable은 이것을 "가장 큰 누락 아이디어"로 꼽았다. 실측은 ** | **매치 스택 (critic ON)** | 131M | **0.6094** (+22.0점) | | 매치 스택 (critic OFF) | 131M | 0.5526 (+10.9점) | | league (기존 최강, 웹 배포판) | 122.6M | — | + +### 기여도 재측정: 동일 규모(131M)에서 + +39.3M A/B는 critic에게 가장 불리한 지점이었다. 두 조각을 **같은 131M 예산**에서 다시 쟀다: + +| 조각 | full의 승률 (그 조각 뺀 버전 상대) | 순이득 | +|---|---|---| +| **양쪽 좌석 학습** | 0.6707 | **+17.1 %p** | +| **전지적 critic** | 0.5367 | **+3.7 %p** | + +**양쪽 좌석이 약 4.6배 크다.** 그리고 성격이 다르다: + +| 규모 | 양쪽 좌석 | 전지적 critic | +|---|---|---| +| 39.3M | +16.8 %p | **−1.6 %p** | +| 131M | **+17.1 %p** | +3.7 %p | + +**양쪽 좌석은 규모와 무관하게 안정적**(+17%p). **critic은 규모를 탄다.** + +이유: 양쪽 좌석은 **데이터를 2배로** 만든다(65.5M → 131M, 같은 컴퓨트에서). critic은 데이터를 +안 늘리고 **채점 정확도만** 올리는데, 코치가 배울 게 많아져서 데이터가 부족하면 오히려 +엉터리 채점을 한다. + +**Fable은 critic을 "가장 큰 누락 아이디어" 1순위로, 양쪽 좌석을 4순위로 꼽았다. 정확히 거꾸로였다.**