Bayesian probability

Part of a series on
Bayesian statistics
Posterior = Likelihood × Prior ÷ Evidence
Background
Model building
  • Weak prior ... Strong prior
  • Conjugate prior
  • Linear regression
  • Empirical Bayes
  • Hierarchical model
Posterior approximation
  • Markov chain Monte Carlo
  • Laplace's approximation
  • Integrated nested Laplace approximations
  • Variational inference
  • Approximate Bayesian computation
Estimators
  • Bayesian estimator
  • Credible interval
  • Maximum a posteriori estimation
Evidence approximation
  • Evidence lower bound
  • Nested sampling
Model evaluation
  • Bayes factor
  • Model averaging
  • Posterior predictive

Bayesian probability figures out the likelihood that something will happen based on available evidence. This is different from frequency probability which determines the likelihood something will happen based on how often it occurred in the past.

You might use Bayesian probability if you don't have information on how often the event happened in the past.

Example

As an example, say you want to classify an email as "spam" or "not spam". One thing you know about this email is that it has an emoji in the subject line. Say it's the year 2017, and 80% of the emails you got with emoji in them were spam. So you can look at an email with emoji in the subject and say it's 80% likely to be spam.

But if only 1% of your emails were spam and 80% of the emojis were spam, that's different than if half your emails are spam and 80% of emoji emails were spam.

Then you can use Bayes's Theorem to determine one probability of whether this email is spam:

p (is_spam | contains_emoji) = [ p(contains_emoji | is_spam) * p(is_spam) ] / p(contains_emoji)

P(xy) = P(xy) * Px / Py