Double descent is a variance spike

Excess test risk of least squares vs. aspect ratio γ = p/n, isotropic Gaussian features, asymptotic limit (Hastie, Montanari, Rosset, Tibshirani 2019).

Hover or touch the plot to read values at a given γ.
SNR r²/σ²
σ² = 1 fixed; risk is in units of σ².
Curves
λ
Decompose (bias² / variance)
bias²   variance (stacked)
Simulated: mean ± 1 s.e. over 8 random designs X.

What to look at. The min-norm curve (exact asymptotic formula, Hastie et al. Thm 1) is σ²γ/(1−γ) for γ<1 and r²(1−1/γ) + σ²/(γ−1) for γ>1. Turn on the decomposition: the bias² is zero below the threshold and grows smoothly after it, so the peak at γ = 1 is entirely the variance term blowing up, because X⊤X/n has eigenvalues near zero when p ≈ n. Any ridge penalty λ > 0 caps that variance, and at the optimal λ* = σ²γ/r² the risk is monotone in γ, with no peak at all. The second descent is real but modest: for SNR > 1 the min-norm risk dips below the null risk r² (minimum at γ = r/(r−σ)), and optimal ridge is at least as good as min-norm at every γ.

Exact vs. simulated. Curves are p, n → ∞ limits, written in closed form through the Marchenko–Pastur Stieltjes transform m(−λ): bias² = r²λ²m′(−λ), variance = σ²γ[m(−λ) − λm′(−λ)]. Dots are finite-sample (n = 100) values: for each of 8 Gaussian designs the risk is computed exactly, averaged over noise and over the direction of β. Near p = n the finite-n min-norm variance is heavy-tailed (for p < n its mean is σ²p/(n−p−1)), so those dots scatter; points above the plot are marked with ▲.

Conventions. Risk R = E[(x₀⊤β̂ − x₀⊤β)² | X] = E‖β̂ − β‖², i.e. excluding the irreducible σ² (as in the paper); ‖β‖² = r²; ridge β̂λ = (X⊤X + nλI)−1X⊤y, i.e. penalty λ‖β‖² on the loss (1/n)‖y − Xβ‖².

What this does not show. This is one linear model with isotropic Gaussian features and a well-specified target. Double descent in deep networks (Nakkiran et al.) is phenomenology, not a consequence of this formula, and it depends on width, label noise and training time in ways this model does not capture. LLM scaling laws do not identify tokens with i.i.d. regression samples, so γ here is not a "parameters per token" ratio.