Probit, logit, sigmoid, softmax, and argmax are often taught as isolated machine-learning vocabulary. They are better read as statistical links, probability transforms, and decision operators.

The useful point is that neural-network output layers are statistical interfaces, not just arbitrary activation functions.

Statistical-modeling noteOriginally posted 2023-08-18; expanded here around output-layer terminology.

Article focus: probit, logit, sigmoid, Gaussian CDF, softmax, argmax, classification, and neural-network outputs.

Links And Inverses

The probit function is the inverse CDF of the standard normal distribution. The logit function is the inverse of the logistic sigmoid. These are link functions: they map probabilities into an unbounded latent scale.

The derivative of the Gaussian CDF is the Gaussian density. The derivative of the sigmoid has its own peaked shape. Those derivatives matter because learning systems optimize through these nonlinear maps.

Softmax And Argmax

Softmax maps a vector of scores into a probability distribution over classes. Argmax selects the index with the largest score or probability. Argmax is not a true inverse of softmax; it is a decision rule applied after scores or probabilities have been computed.

That distinction matters in neural networks. Training usually needs differentiable probability-like outputs; deployment may collapse those outputs into a discrete class choice.

The Neural-Network Point

A deep network output layer often acts as a statistical model head. Attention, hidden layers, and embeddings produce representation; the final probability interface determines how the result is interpreted and trained.

That is why neural networks are closer to statistics than the word "perceptron" suggests.