A visual spectral reconstruction

Past × future reveals the hidden dimension.

Start with one visible symbol stream. Arrange its word probabilities into a Hankel matrix, keep r singular directions, and reconstruct a rank-r predictive machine.

Toy process · computed
Generator
What we observe

True HMM

shown because this is a toy
What the learner gets

Observed Hankel block

H[p,s] = P(ps)
Singular spectrum

Reconstructed predictive machines

small multiples

These are WFA/PSR coordinates. Solid edges are positive operator entries; dashed edges are negative. A good linear realization need not already be a stochastic HMM.

Error across every rank

r = 1…6
How to read the spectrum. Population rank and sample rank are different claims. Exact probabilities stop at the process rank; a finite-sample Hankel block is generically higher rank, often full rank, with a small noise tail below the signal.
About “Hankel error.” The fourth curve is the relative Frobenius residual of this finite block. The spectral-norm residual is instead σr+1. Neither should be silently conflated with KL or word-probability error.