← All papers
First page of Equal Steps, Unequal Curvature: Where Muon's Update Goes Wrong

Equal Steps, Unequal Curvature: Where Muon's Update Goes Wrong

Biswa Sengupta, Jinhua Wang, Léo Maxime Brunswic

cs.LG Nov 18, 2026 · v1
Every theorem and proposition about Muon's update, Newton–Schulz iteration, block polar factors and second-order criteria is proved in Lean 4 with Mathlib.
Muon updates each hidden weight matrix along the polar factor of its momentum, the steepest-descent direction under the spectral norm. That choice fixes two allocations without consulting the loss: every singular direction of the momentum receives the same step, and in a non-square matrix each neuron's step is set by its leverage score. Whether they are good is a question about curvature, which steepest-descent arguments do not answer; we measure it. Using held-out gradients and Hessian-vector products, we decompose Muon's step by singular-value band and by neuron group and compute each group's second-order optimal multiplier. In a 124M-parameter decoder trained on FineWeb-Edu, the multipliers rise monotonically from the top singular directions to the tail: Muon over-steps the high-curvature top of its spectrum. We prove in Lean when such a diagnostic licenses an intervention at Muon's own update norm. On the measured curvature, the criterion predicts, and held-out losses confirm, that moving step length off the top helps mid-training and early in the cooldown, but not at its end. We then test interventions under per-step norm matching, tuned baselines and paired seeds. Orthogonalizing Q, K and V separately rather than as one fused matrix lowers the loss by 0.0250 nats and accounts for all of the gain of block-wise variants. A matmul-only, rotation-equivariant spectral tilt $σ^{-1/4}$ applied in the cooldown lowers the loss of this stronger baseline by a further 0.0120 nats, additively with NorMuon (0.0158 nats together, three seeds; the baseline needs about 4.3% more tokens to match), and by 0.0188 nats at 355M parameters (three seeds). Energy-matched Schatten-1 weight decay adds a further 0.0052 nats. Geometrically principled changes that do not act on the measured miscalibration, namely exact polar factors, steepest-descent input whitening and top-heavy decay, do not help.

Muon steps along the polar factor of the momentum. This gives every singular direction an equal step and sets each neuron's step by leverage scores, without consulting the loss curvature. Whether these allocations are good is a curvature question that steepest-descent theory does not answer.

Held-out gradients and Hessian-vector products are used to decompose Muon's step by singular-value band and by neuron group, and to compute second-order optimal multipliers. A family of supporting results is proved in Lean 4 with Mathlib in exact real arithmetic. These include a scale/shape decomposition of the second-order gap, a fixed-norm intervention criterion, and Newton–Schulz and block energy identities. Interventions such as split-QKV orthogonalization, a cooldown spectral tilt σ^{-1/4}, NorMuon and Schatten-1 decay are tested under norm matching with paired seeds.

Multipliers rise monotonically from the top singular directions to the tail, so Muon over-steps its high-curvature top directions. Orthogonalizing Q, K and V separately lowers loss by 0.0250 nats. The cooldown tilt adds a further 0.0120 nats, and 0.0188 nats at 355M parameters. Formal verification fixed bookkeeping and the authors' own reasoning but did not predict training benefit.

OptimizerLossΔ vs. split-QKV
Muon (fused QKV)3.9427+0.0250
Split-QKV Muon3.9176—
+ cooldown tilt3.9057-0.0120
+ NorMuon + tilt3.9018-0.0158
+ NorMuon + tilt + Schatten-1 decay3.8966-0.0210
Final held-out loss, 124M model (selected rows)