Original Post

Probit is inverse of cdf of Gaussian curve. Argmax is inverse of softmax. Logit is inverse of the sigmoid. Derivative of the the cdf is the Gaussian curve. The derivative of the softmax and the sigmoid both also look bell-shaped. What are really neural networks are doing, in addition to attention? The perceptron in deep networks is not actually what you think it is. NN are much closer connected to statistics than we teach other.