Hilbert 13 and Neural Networks
April 18, 2022

Machine-learning noteOriginally posted 2022-04-18; rewritten here around multivariate approximation and scaling.

Included: Hilbert's 13th problem as motivation, neural networks, sum-of-products structure, Q-learning, Monte Carlo methods, and the gap between empirical behavior and mechanism.

Approximation and Scaling

The useful question in this note is how multivariate functions become representable at scale. Neural networks, sum-of-products forms, and related approximators all ask a practical version of the same question: how can many interacting variables be represented without the expression becoming unusable?

Hilbert's 13th problem is useful here as a historical pointer toward composition and representation. The modern engineering question is not only whether a network can fit a function, but what structure makes the fit efficient, trainable, and inspectable enough to use.

Learning Systems Still Need Mechanism

Q-learning, deep networks, recursive structures, Monte Carlo methods, and reinforcement-learning systems can work impressively in practice. That empirical success does not remove the need to understand stability, sample cost, state representation, discontinuities, and feedback paths.

The durable point is caution and curiosity together: when a method works better than the simple mental model predicts, the next useful step is to identify the mechanism, limits, and operating conditions.

Useful Scope

This is a short approximation note, not a claim that arbitrary functions, quantum machine learning, or reinforcement learning are solved. It points to a real engineering question: how to keep representation, training, and mechanism attached while model capacity scales.