Functions, Prediction, and Autoregression

A function maps arguments to values. A regression model estimates that map from data. An autoregressive model uses earlier values as part of the input for the next value.

Computers already contain several ways to build this behavior: Boolean functions, finite-state machines, event-driven datapaths, statecharts, CPUs, and GPUs. Each representation exposes a different handle on state, timing, and control.

Transformers are still functions in the mathematical sense, but their inputs include context, token history, position, learned embeddings, and attention weights. That makes the generated output less like a simple lookup and more like a computation over assembled state.

The engineering question is where that state lives, how it is updated, how much context is retained, and what the model is actually conditioned on when it predicts the next token.