Add diagnostic value-head metrics: rmse, target stats
Codex flagged that mcts/value_prediction_error mathematically reconciles with loss/value (MSE / value_scale^2 = 0.05) but the latter looks healthy while the former says the value head is far off. To make this clearer in W&B, expose: - mcts/value_rmse — sqrt(MSE), in raw score units (interpretable) - mcts/v_target_abs_mean — magnitude of |v_target|, indicates if game outcomes are very lopsided (always negative for a losing agent) - mcts/v_target_std — spread, low std means targets are saturated to one end (e.g., always -100ish) These let us see whether value head is failing because targets are unlearnable variance, or just hard-to-predict, or because of saturation at the value_scale=100 tanh boundary. Tests: 19/19 passing.
This commit is contained in:
@@ -365,9 +365,15 @@ class IsMctsTrainer:
|
|||||||
with torch.inference_mode():
|
with torch.inference_mode():
|
||||||
_logits, value_pred = self.network(info, legal)
|
_logits, value_pred = self.network(info, legal)
|
||||||
value_error = nn.functional.mse_loss(value_pred, target)
|
value_error = nn.functional.mse_loss(value_pred, target)
|
||||||
|
value_rmse = float(value_error.item()) ** 0.5
|
||||||
|
target_abs_mean = float(target.abs().mean().item())
|
||||||
|
target_std = float(target.std().item()) if target.numel() > 1 else 0.0
|
||||||
return {
|
return {
|
||||||
"mcts/avg_visit_entropy": float(np.mean(entropies)) if entropies else 0.0,
|
"mcts/avg_visit_entropy": float(np.mean(entropies)) if entropies else 0.0,
|
||||||
"mcts/value_prediction_error": float(value_error.item()),
|
"mcts/value_prediction_error": float(value_error.item()),
|
||||||
|
"mcts/value_rmse": value_rmse,
|
||||||
|
"mcts/v_target_abs_mean": target_abs_mean,
|
||||||
|
"mcts/v_target_std": target_std,
|
||||||
"mcts/policy_mcts_kl": float(np.mean(policy_kls)) if policy_kls else 0.0,
|
"mcts/policy_mcts_kl": float(np.mean(policy_kls)) if policy_kls else 0.0,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user