Winning the head-to-head says a policy is strong on average, not that it is hard
to beat. So freeze each policy, train a fresh one from scratch whose only job is
to beat that policy, and see how far it gets.
Same exploiter budget (250 updates x batch 1024, 65.5M learner actions):
ours (match stack, 131M) exploiter reaches 0.2278 [0.219, 0.237]
league (web-deployed) exploiter reaches 0.3213 [0.311, 0.331]
League gives up 9.4 more points to a dedicated attacker. Both sit far from 0.5, so
neither is a pushover -- but the caveat left open by the head-to-head is now closed
in our favour on both axes: stronger on average (0.6094) and harder to exploit.
Worth noting against expectation: league was trained *with* an exploiter structure
and we ran pure self-play, and we still come out less exploitable. Whatever the
league machinery buys, it did not buy that here.
The number is a lower bound -- a bigger exploiter would find more -- so it only
means anything as a like-for-like comparison, which is how it is used.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
coolguyandClaude Opus 4.8
e51d14d0fb