Autoregression, Nonlinear Processes, and Embedding

Autoregression predicts future values from earlier values. For nonlinear processes, the hard question is not only the regression rule; it is what state representation makes the process predictable.

Embedding theorems matter because they show how the state of a dynamical system can be represented through observations over time. That gives a clean way to think about sequence models: previous samples are not just history; they can carry enough structure to reconstruct useful state.

A transformer embedding is not the same object as a dynamical-systems embedding, but the family resemblance is useful. Both ask how raw observations become coordinates in a space where computation becomes possible.

The practical point is modest: sequence models inherit older mathematical questions about state, delay, representation, and prediction.