coolguyandClaude Opus 4.7
004b913a7b
Three coordinated hygiene changes; none target the diagnosed
selection-bias bottleneck. They make the codebase honestly reflect the
pure-self-play stance and reduce dashboard noise.
Bot rename (drop the unhelpful safe_ prefix; suffixes describe behaviour):
- safe_heuristic_loose -> heuristic_aggressive
- safe_heuristic -> heuristic_balanced
- safe_heuristic_strict -> heuristic_cautious
- noisy_safe -> heuristic_noisy
- passive_discard -> discard_only
Class renames in bots/: SafeHeuristicBot -> HeuristicBot,
SafeHeuristicParams -> HeuristicParams, PassiveDiscardBot -> DiscardOnlyBot,
plus loose/strict parameter constants. Backwards compatibility was dropped
intentionally per user instruction; no aliases. Active configs, docs,
scripts, tests updated. Archive directories (configs/archive,
docs/archive, runs/archive) left intact and may still reference old
names per their read-only policy. The src/.../bots/passive.py module was
renamed to discard_only.py via git mv.
Analyze plot curation (deep_cfr/analyze.py):
- Added analysis_00_core.png as the canonical daily dashboard with 10
heuristic-free metrics (loss/{advantage,strategy}; vs heuristic_cautious:
avg_score_diff0, win_rate0, avg_opened_colors, positive_expedition_rate,
bonus_expedition_rate, score_per_opened_color, policy_entropy; vs random:
win_rate0).
- Removed analysis_05_open_quality.png (bad/weak/good open rates,
recoverable score) and analysis_07_calibration.png (calibration gap,
recoverable mean) - both relied on the heuristic recoverable_score
classifier already dropped from inputs.
- Removed SELECTIVITY_PLOTS and plot_selectivity (heuristic-laden).
- SUMMARY_EVAL_METRICS no longer includes bad_open_rate or
calibration_gap.
- PlotSpec gained an opponents allowlist so the new core section can pin
a specific opponent per panel without restructuring plot_section.
Tiered evaluation cadence (EvaluationConfig):
- Added extended_opponents and extended_eval_every (default 0 = disabled).
- opponents_for_iteration(iteration) returns the core list every
eval_every and appends extended_opponents (de-duplicated) when
iteration is also a multiple of extended_eval_every.
- default.yaml now uses 3 core opponents (random, discard_only,
heuristic_cautious) every 5 iterations and 3 extended opponents
(heuristic_balanced, heuristic_aggressive, heuristic_noisy) every 50
iterations. random is the floor sanity. discard_only is the
zero-pit detector / absolute-score reference (its score is always 0,
so eval/discard_only/avg_score_diff0 directly equals the model's raw
average score). heuristic_cautious is the ceiling and the
archive-comparable benchmark used in the prior diagnostic sections.
Net eval cost reduction: roughly 50% (3 opponents x every 5 iter, plus
6 opponents x every 50 iter, vs the prior 6 x every 5).
Documented in docs/plans/deep-cfr-selectivity.md section 9.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
110 lines
2.1 KiB
YAML
110 lines
2.1 KiB
YAML
run:
|
|
experiment_name: deep-cfr-default-server
|
|
seed: 79
|
|
max_iterations: 1000
|
|
max_minutes: null
|
|
device: cuda
|
|
use_amp: false
|
|
deterministic: false
|
|
|
|
rules:
|
|
n_colors: 5
|
|
n_ranks: 9
|
|
min_rank: 2
|
|
n_handshakes: 3
|
|
hand_size: 8
|
|
expedition_penalty: -20
|
|
bonus_threshold: 8
|
|
bonus_amount: 20
|
|
|
|
encoding:
|
|
derived_playability: true
|
|
slot_aware_playability: true
|
|
|
|
network:
|
|
hidden_size: 512
|
|
num_layers: 3
|
|
activation: relu
|
|
|
|
traversal:
|
|
traversals_per_player: 280
|
|
max_depth: null
|
|
max_nodes_per_traversal: 1000
|
|
regret_matching_epsilon: 0.0001
|
|
outcome_sampling_epsilon: 0.2
|
|
outcome_sampling_value_clip: 500.0
|
|
outcome_unsampled_regret: zero
|
|
cutoff_value_mode: score_diff
|
|
cutoff_rollouts: 0
|
|
cutoff_rollout_policy: random
|
|
cutoff_rollout_max_steps: 300
|
|
opponent_policy: average_strategy
|
|
strategy_sample_interval: 1
|
|
store_strategy_on_traverser_nodes: true
|
|
store_strategy_on_opponent_nodes: false
|
|
num_workers: 8
|
|
worker_chunk_size: 8
|
|
progress_every_traversals: 10
|
|
endpoint_depth_bucket_width: 100
|
|
endpoint_depth_bucket_max: 1000
|
|
inference_backend: server
|
|
|
|
regret_matching:
|
|
all_negative_fallback: argmax_tiebreak
|
|
|
|
training_weighting:
|
|
mode: lcfr
|
|
|
|
self_play:
|
|
snapshot_every: 1
|
|
max_snapshots: 0
|
|
anchor_probability: 0.0
|
|
current_weight: 1.0
|
|
recent_weight: 0.0
|
|
older_weight: 0.0
|
|
anchor_weight: 0.0
|
|
recent_window: 5
|
|
|
|
optimization:
|
|
advantage_updates_per_iteration: 512
|
|
strategy_updates_per_iteration: 512
|
|
advantage_batch_size: 1024
|
|
strategy_batch_size: 1024
|
|
learning_rate: 1.0e-4
|
|
weight_decay: 0.0001
|
|
grad_clip: 1.0
|
|
|
|
memory:
|
|
advantage_capacity: 2000000
|
|
strategy_capacity: 2000000
|
|
|
|
checkpoint:
|
|
save_latest: true
|
|
save_every: 50
|
|
progress_interval_seconds: 20.0
|
|
exact_resume: false
|
|
|
|
evaluation:
|
|
eval_every: 5
|
|
games: 100
|
|
opponents:
|
|
- random
|
|
- discard_only
|
|
- heuristic_balanced
|
|
- heuristic_aggressive
|
|
- heuristic_cautious
|
|
- heuristic_noisy
|
|
max_steps: 10000
|
|
on_max_steps: score_diff
|
|
batch_size: 64
|
|
device: trainer
|
|
num_workers: 4
|
|
|
|
inference_server:
|
|
device: cuda
|
|
num_slots: null
|
|
max_batch: 256
|
|
batch_window_us: 200
|
|
weight_sync_every: 1
|
|
use_amp: false
|