Probit, Argmax, Logit, and Neural Networks
August 18, 2023
Neural-network outputs inherit a deep statistical lineage. Probit connects probabilities to Gaussian quantiles, logit connects probabilities to log-odds, sigmoid and softmax map scores into probabilities, and argmax selects the winning coordinate.
Brian Greenforest uses those relationships to make perceptrons, output layers, attention, and learned classification feel like one continuous mathematical system.
Maps and Choices Have Different Jobs
The probit function inverts the standard normal cumulative distribution. The logit inverts the logistic sigmoid. Softmax converts a vector of logits into a categorical probability distribution.
Argmax does not reverse softmax; it discards probability detail and returns the index with the greatest score. Derivatives of sigmoid and Gaussian CDFs produce bell-shaped sensitivity curves, while the softmax Jacobian couples every category.
Teach Neural Networks Through Statistics
Output layers encode assumptions about outcomes, likelihoods, and geometry. Those choices determine the loss function, gradients, calibration, and the meaning of a prediction.
Statistics educators and ML practitioners can teach these transforms together, derive their gradients, and show how classical links become executable neural architecture.
Run Probit, Argmax, Logit, and Neural Networks in a Four-Layer Transformer
The complete training run connects this mathematical argument to executable code, data flow, and a working small model.
LinkedIn status when archived: Visible to anyone on or off LinkedIn.
Probit is inverse of cdf of Gaussian curve. Argmax is inverse of softmax. Logit is inverse of the sigmoid. Derivative of the the cdf is the Gaussian curve. The derivative of the softmax and the sigmoid both also look bell-shaped. What are really neural networks are doing, in addition to attention? The perceptron in deep networks is not actually what you think it is. NN are much closer connected to statistics than we teach other.
