The exploiters ran 32.5M learner actions, not the 65.5M I recorded -- they train
one seat, so a 250x1024 run yields half of what the same shape gives the
both-seat self-play trainer. Against targets trained on 131M and 122.6M, that
makes the attacker roughly 4x underfunded.
Which means the absolute number does not support "ours is only 22.8%
exploitable". It supports exactly one claim: at a matched budget, league gives up
more. Whether the ordering survives a properly funded attacker is now the open
question, so the script takes --updates and --batch-games to run it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
Winning the head-to-head says a policy is strong on average, not that it is hard
to beat. So freeze each policy, train a fresh one from scratch whose only job is
to beat that policy, and see how far it gets.
Same exploiter budget (250 updates x batch 1024, 65.5M learner actions):
ours (match stack, 131M) exploiter reaches 0.2278 [0.219, 0.237]
league (web-deployed) exploiter reaches 0.3213 [0.311, 0.331]
League gives up 9.4 more points to a dedicated attacker. Both sit far from 0.5, so
neither is a pushover -- but the caveat left open by the head-to-head is now closed
in our favour on both axes: stronger on average (0.6094) and harder to exploit.
Worth noting against expectation: league was trained *with* an exploiter structure
and we ran pure self-play, and we still come out less exploitable. Whatever the
league machinery buys, it did not buy that here.
The number is a lower bound -- a bigger exploiter would find more -- so it only
means anything as a like-for-like comparison, which is how it is used.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
coolguyandClaude Opus 4.8