Probability and Statistics

Log Probability

Since probabilities of independent events multiply, and log converts multiplication to addition, log probabilities of independent events add.

Super useful in research, I see it everywhere.

Note

Taking the exponential of a value in log-space will take it out (recall inverse of natural log), e.g.,

np.log(1)
>> 0
np.exp(0)
>> 1

Motivation

1. Speed. Multiplication more expensive than addition.
2. Accuracy. Improves the numerical stability, when the probabilities are very small, because of floating point arithmetic.
3. Simplicity. Log of probability distribution removes the exponential function.
4. Optimization. Optimizers work better with log probabilities.