coolguy/coorl-lost-cities
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Keep the human game records, and name the models

Browse Source 
The 111 games are the user's own strategy record, not ML data -- so they stay as
they are, and hint/undo usage deliberately goes unlogged. What they were missing
is which model they were played against: the v1 schema stores `policy: "WASM ·
FINAL PPO"`, which is the string shown in the corner of the screen, not an
identity. It stops meaning anything the moment there is a second final model,
which there now is.

data/models.json fixes that going forward. Codenames are astronomical and
alphabetical, so the first letter is the generation and you can always tell which
came later; a new letter means the observation space broke, not that the model got
better. A codename never encodes quality -- that is the trap "FINAL PPO" fell into.
The hash stays the truth, because a hash cannot lie about what actually played;
the codename is assigned here, not derived.

  altair    (e8241e305c01) -- single round, 454-dim, the league policy on the web.
                              All 111 v1 games were played against this.
  borealis  (4ae613b010ca) -- three-round match, 501-dim, 131M. Beats altair
                              0.6094 at the real game and holds up better under a
                              funded exploiter.

The records were also sitting in runs/tmp/, which is gitignored and documented as
disposable. They are in data/ now, and committed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XBQKgvBbxbheiTF1AVy1Sh
This commit is contained in:
coolguy
2026-07-15 06:25:29 +09:00
co-authored by Claude Opus 4.8
parent 0fb6e3c1a6
commit 8dec6c3fe1
2 changed files with 154 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
data/models.json
+43
View File
@@ -0,0 +1,43 @@
{
"_scheme": {
"naming": "Astronomical names, alphabetically ordered. The first letter is the generation; a new letter means the observation space broke, not that the model got better.",
"rule": "A codename never encodes quality. The record this replaces stored 'FINAL PPO', which stops meaning anything the moment there is a second final model.",
"identity": "The hash is the truth -- it is what actually played. The codename is for humans, and it is assigned here, not derived. Records should carry the hash; look the codename up.",
"next": "cygnus, deneb, ..."
},
"models": {
"altair": {
"hash": "e8241e305c01",
"hash_kind": "sha256 of web/public/models/jax-ppo.onnx",
"generation": "a",
"game": "single round",
"observation_size": 454,
"hidden_size": 512,
"num_layers": 3,
"trained": "self-play league, 122.6M learner actions",
"source": "/mnt/2tbhdd/coolrl-lost-cities-artifacts/league/2026-07-05_052325_jax-ppo-league-v1/latest",
"deployed": "web/public/models/jax-ppo.onnx",
"displayed_as": "WASM · FINAL PPO",
"note": "Every game in data/human-play/game-records.jsonl (format v1, 111 games, 2026-07-14) was played against this model. The v1 schema has no model field -- it stores the on-screen label -- so this line is the record of what they played."
},
"borealis": {
"hash": "4ae613b010ca",
"hash_kind": "sha256 over the orbax checkpoint files (no ONNX export yet)",
"generation": "b",
"game": "three-round match (classic rules)",
"observation_size": 501,
"critic_observation_size": 681,
"hidden_size": 512,
"num_layers": 3,
"trained": "self-play, 131.1M learner actions, linear total-score reward, both seats, privileged critic",
"source": "runs/jax-ppo-match/2026-07-15_031529_match-scaled/latest",
"deployed": null,
"results": {
"vs_altair_3round": "0.6094 win rate [0.599, 0.620], +22.0 points, 8192 duplicate matches",
"exploitability": "a from-scratch exploiter funded to 131M reaches 0.4657 against it, and 0.6295 against altair -- lower bound, neither exploiter had plateaued"
}
}
}
}