What Is Bayes' Theorem?
Bayes' Theorem is one rule that tells you how to revise a probability when you receive new information. It is the engine behind medical diagnosis, spam filters, machine learning classifiers, modern AI, and any rational update of belief from evidence.
P(A | B) = P(B | A) · P(A) / P(B). In words: the probability of A given that B happened equals the probability of B given A, times the unconditional probability of A, divided by the unconditional probability of B.Naming the four pieces
Each piece has a name. P(A) is the prior — what you believed about A before seeing the evidence. P(B | A) is the likelihood — how probable the evidence B is when A is true. P(B) is the total probability of the evidence, sometimes called the marginal likelihood. P(A | B) is the posterior — the updated probability of A after observing B.
The genius of the theorem is that it lets you flip a conditional probability. You often know P(B | A) ("if the patient has the disease, the test is positive 95% of the time") and want P(A | B) ("if the test is positive, what is the probability the patient has the disease?"). Those two numbers are almost never equal — and Bayes' theorem is the bridge between them.
The Intuition — Updating Beliefs With Evidence
Bayes' theorem is best understood as a recipe for changing your mind in a principled way. Start with a prior belief about how likely some hypothesis is. Then observe some evidence. Bayes tells you exactly how to combine the prior with the evidence to get a posterior belief — your updated probability after the evidence.
Three pieces drive the update
- Strength of the prior
If you started out very confident (or very skeptical), it takes a lot of evidence to move you much.
- The likelihood ratio
How much more likely is the evidence under the hypothesis than against it? Strong evidence is evidence that is much more likely under one hypothesis than the other.
- The base rate of the evidence
If the evidence is common regardless of the hypothesis, it doesn't tell you much.
Suppose 1% of emails are spam. A spam filter flags 99% of spam correctly, but also flags 5% of legitimate emails by mistake. An email arrives and the filter flags it. What is the probability it is actually spam?
Intuitively many people say "99%" because the filter is 99% accurate on spam. But that ignores the base rate.
Working through Bayes: P(spam | flagged) = (0.99 × 0.01) / (0.99 × 0.01 + 0.05 × 0.99) = 0.0099 / 0.0594 ≈ 16.7%.
P(spam | flagged) ≈ 16.7%Most flagged emails are legitimate, simply because legitimate emails massively outnumber spam. This is a small numeric statement with huge real-world consequences — most of medical screening, fraud detection, and AI-classifier confidence rests on it.
Deriving Bayes' Theorem from Conditional Probability
Bayes' theorem isn't a new axiom — it falls out of the definition of conditional probability in two lines.
- Definition of conditional probability
P(A | B) = P(A and B) / P(B)(wheneverP(B) > 0). Rearranging:P(A and B) = P(A | B) · P(B). - Use symmetry of the joint
By symmetry,
P(A and B) = P(B | A) · P(A)as well — the joint probability ofAandBdoesn't care which way you condition. - Equate and divide
Setting the two right-hand sides equal:
P(A | B) · P(B) = P(B | A) · P(A). Divide both sides byP(B):P(A | B) = P(B | A) · P(A) / P(B). That's Bayes' theorem.
Expanding the denominator with the law of total probability
In most real problems you don't know P(B) directly. You usually know P(B | A) and P(B | not A). Total probability says P(B) = P(B | A) · P(A) + P(B | not A) · P(not A) — the evidence happens either with A or without it, weighted by how likely each is.
P(A | B) = [P(B | A) · P(A)] / [P(B | A) · P(A) + P(B | not A) · P(not A)].The general form with multiple hypotheses
A₁, A₂, …, Aₙ is a partition of the sample space (mutually exclusive and exhaustive), then for any specific Aᵢ: P(Aᵢ | B) = [P(B | Aᵢ) · P(Aᵢ)] / Σⱼ [P(B | Aⱼ) · P(Aⱼ)]. This is the version used in machine learning — Naive Bayes classifiers pick the class Aᵢ that maximizes the numerator (the denominator is the same for every class).The Classic Medical Test Example — Why Doctors Get This Wrong
The single most famous Bayes problem is the rare-disease screening test, and it's worth working through carefully because the answer surprises everyone the first time.
Setup. A disease affects 1 in 1,000 people. There is a test for it with the following properties: if you have the disease, the test is positive 99% of the time (sensitivity = 0.99). If you don't have the disease, the test is positive 5% of the time anyway (false positive rate = 0.05). You test positive. What is the probability you actually have the disease?
The intuitive (wrong) answer: most people say 99%, or at least "very high."
The Bayes answer. Let D = has disease, + = positive test. We want P(D | +). Knowns: P(D) = 0.001, P(not D) = 0.999, P(+ | D) = 0.99, P(+ | not D) = 0.05.
Apply Bayes: P(D | +) = (0.99 × 0.001) / (0.99 × 0.001 + 0.05 × 0.999) = 0.00099 / (0.00099 + 0.04995) = 0.00099 / 0.05094 ≈ 0.0194 = 1.94%.
P(D | +) ≈ 1.94%Even with a positive test, the probability you have the disease is under 2%. The disease is so rare that the much larger pool of healthy people producing a 5% false positive rate swamps the small number of true positives from sick people. Out of 100,000 random people: about 100 have the disease and ~99 test positive. About 99,900 don't have it and ~4,995 test positive. So a random positive test is one of ~5,094 positives, only ~99 of which are real — about 1.9%.
Why doctors get this wrong
Multiple studies (the most famous by Gigerenzer) have shown that even medical professionals routinely confuse P(+ | D) — the test's sensitivity — with P(D | +) — the post-test probability of disease. The first is a property of the test. The second is what the patient actually wants to know, and it depends critically on the prior probability of disease. This confusion is a special case of the base rate fallacy — ignoring the prior when interpreting evidence.
Bayes in Action — Spam Filters, Diagnostics, and Court Cases
Bayes' theorem is not a textbook curiosity — it is woven into systems you interact with every day.
Spam filters
Naive Bayes classifiers were the first widely-deployed spam filters and are still competitive. The filter learns, for many individual words, P(word | spam) and P(word | not spam). For each incoming email, it treats the words as independent evidence (the "naive" assumption) and combines their likelihoods to produce a posterior P(spam | words observed). If that posterior exceeds a threshold, the email is flagged. Despite the independence assumption being literally false (English words are correlated), Naive Bayes works remarkably well in practice because the ranking it produces is robust even when the absolute probabilities are off.
Medical diagnostics
Every diagnostic test — pregnancy, HIV, COVID, mammography, PSA — has a published sensitivity and specificity. Combined with the prevalence in the relevant population (the prior), Bayes gives the post-test probability. This is why mass screening of low-prevalence populations is controversial: even highly accurate tests produce mostly false positives when the true disease rate is very low.
Court cases
The prosecutor's fallacy is a Bayes error that has overturned convictions. A prosecutor presents DNA evidence with a 1-in-a-million random match probability — P(DNA match | innocent) = 10⁻⁶ — and claims this means the probability the defendant is innocent is 1 in a million: P(innocent | match) = 10⁻⁶. But those two probabilities are different. If a city of 5 million people is screened and you get a match, you would expect about 5 random matches by chance, so the probability that any one matched person is the culprit could be much lower than 1 in a million depending on other evidence. Several wrongful convictions (R v Adams in the UK, the Lucia de Berk nursing case in the Netherlands) hinged on exactly this confusion.
Machine learning and AI
Bayes' theorem underlies Bayesian networks, probabilistic programming, Markov chain Monte Carlo, and the entire field of Bayesian inference. Modern language models do not use Bayes directly, but the broader idea of treating learning as posterior updating from data remains central to the theory of machine learning.
Common Mistakes and Pitfalls
Bayes problems are mechanical once you set them up correctly — but the setup is where students lose points.
P(A | B) with P(B | A)This is the single most common error and the root of both the base rate fallacy and the prosecutor's fallacy. Always read the problem carefully and write down which conditional probability you are given and which you want. They are almost never the same number.
If a test is "99% accurate" but the condition is rare, most positives are still false. The prior P(A) is part of the formula for a reason.
Students often plug in P(B | A) without recognizing that P(B) = P(B | A) · P(A) + P(B | not A) · P(not A) when P(B) isn't given directly. Skipping this step gives nonsensical answers like posteriors greater than 1.
Mixing 99% and 0.99 in the same equation produces off-by-100 errors. Pick one convention (decimals are safer) and stick with it.
They don't. Sensitivity is P(+ | disease), specificity is P(− | no disease). The false positive rate is 1 − specificity. Always double-check which one the problem is giving you.
Naive Bayes assumes evidence pieces are independent given the class. If they aren't, you can wildly overcount the strength of correlated evidence. For exam problems this rarely comes up, but it is the main failure mode of Naive Bayes in practice.
A and not A halfway throughDefine your hypotheses clearly at the top of the page and don't change notation midstream. "Probability the defendant is guilty" and "probability the defendant is innocent" must add to 1 — verify this at the end of every problem.
Worked Examples and Bayesian vs Frequentist Thinking
Two practice problems and a note on the broader philosophical context.
Jar 1 has 30 vanilla and 10 chocolate cookies. Jar 2 has 20 vanilla and 20 chocolate cookies. You pick a jar at random (50/50) and draw a vanilla cookie. What is the probability you picked Jar 1?
Solution. Let A = picked Jar 1, B = drew vanilla. Prior: P(A) = 0.5, P(not A) = 0.5.
Likelihoods: P(B | A) = 30/40 = 0.75, P(B | not A) = 20/40 = 0.5.
Bayes: P(A | B) = (0.75 × 0.5) / (0.75 × 0.5 + 0.5 × 0.5) = 0.375 / 0.625 = 0.6.
P(A | B) = 0.6The vanilla cookie raised our confidence from 50% to 60% that we picked Jar 1.
A factory produces widgets, and 2% are defective. An inspection catches 95% of defective widgets but also flags 10% of good widgets. A widget is flagged. What is the probability it is actually defective?
Solution. Let D = defective, F = flagged. Knowns: P(D) = 0.02, P(F | D) = 0.95, P(F | not D) = 0.10.
Total probability of flag: P(F) = 0.95 × 0.02 + 0.10 × 0.98 = 0.019 + 0.098 = 0.117.
Posterior: P(D | F) = 0.019 / 0.117 ≈ 0.162 = 16.2%.
P(D | F) ≈ 16.2%Most flagged widgets are good — the inspector's false-positive volume swamps the true positives.
Bayesian vs frequentist
Bayes' theorem is mathematically uncontroversial — every statistician accepts it. The deeper debate is whether you should treat probability as a measure of belief that gets updated with evidence (the Bayesian view, where priors are explicit) or strictly as a long-run frequency (the frequentist view, which avoids priors). Confidence intervals and p-values come from the frequentist tradition; credible intervals and posterior distributions come from the Bayesian one. Modern data science uses both: Bayesian methods dominate in machine learning, A/B testing platforms, and reinforcement learning; frequentist methods dominate in classical hypothesis testing and regulatory science.
When you have a prior — a base rate, prevalence, or honest prior belief — Bayes' theorem is the only correct way to combine it with new evidence. Ignore the prior and you get the base rate fallacy. Use the prior incorrectly and you get the prosecutor's fallacy. Get it right and you have one of the most powerful tools in all of probabilistic reasoning. When working through Bayes problems for a stats course or AP Statistics prep, scan them with Solver AI for the full breakdown of prior, likelihood, total probability, and posterior calculation.