Cross-Entropy Intuition
September 19, 2023

Cross-entropy is easiest to read as an expectation under the target distribution of the negative log probability assigned by the model.

That connection explains why the target enters the loss and why the same expression appears as negative log likelihood in classification training.

Mathematical noteOriginally posted 2023-09-19; expanded here around the loss expression.

Article focus: cross-entropy, negative log likelihood, target distribution, model distribution, expectation, and classification loss.

Related: Four-Layer Tiny Transformer Training Run

The Loss

For a target distribution p and model distribution q, cross-entropy is the expected value of -log q under p. In a one-hot classification target, that reduces to the negative log probability assigned to the correct class.

That is why the same training expression is often called negative log likelihood.

Why The Target Matters

The expectation is taken with respect to the target distribution. The model distribution supplies the probabilities being scored, but the target distribution says which outcomes matter for the sample or dataset.

This is the useful intuition: the loss punishes the model when it assigns low probability to outcomes the target says should be likely.

Where Matrix Exponentials Fit

Softmax contains exponentials, and matrix exponentials appear in other mathematical settings, but cross-entropy itself does not require a quantum or eigendecomposition interpretation. The classification case is already useful: logits become probabilities, and the loss scores those probabilities against the target.

Links From the Original Post