Introduction to Bayes’ Theorem

 

Introduction to Bayes’ Theorem

Bayes’ Theorem is a fundamental concept in probability theory that describes how to update our beliefs when new information becomes available. It provides a mathematical way to revise existing predictions or hypotheses in light of evidence.

At its core, it answers this question:

“Given that we have observed some evidence, how should we adjust the probability of a hypothesis?”

---

🔢 The Formula

$P(A\mid B)=\frac{P(B\mid A)\,P(A)}{P(B)}$

🧠 Understanding the Components

• Prior $P(A)$
  Your initial belief about an event or hypothesis before seeing any data.
• Likelihood $P(B \mid A)$
  The probability of observing the evidence assuming the hypothesis is true.
• Evidence $P(B)$
  The total probability of observing the evidence under all possible hypotheses.
• Posterior $P(A \mid B)$
  The updated probability of the hypothesis after considering the evidence.

---

💡 Intuition

Bayes’ Theorem is essentially a learning rule:

Start with a belief → Observe data → Update the belief

It allows us to move from prior knowledge to informed conclusions.

---

🎯 A Clear Example of Bayes’ Theorem (Medical Testing)

Let’s walk through a classic, intuitive example you can use in class.

---

🧪 Problem Setup

Suppose:

• 1% of people have a certain disease
  → $P(D) = 0.01$
• A test is 99% accurate:
  • If a person has the disease → test is positive 99% of the time
    → $P(+ \mid D) = 0.99$
  • If a person does not have the disease → test is positive 5% of the time (false positives)
    → $P(+ \mid \neg D) = 0.05$

---

❓ Question

If a person tests positive, what is the probability they actually have the disease?

---

🔢 Apply Bayes’ Theorem

$P(D\mid +)=\frac{P(+\mid D)\,P(D)}{P(+)}$

---

Step 1: Compute Total Probability of Positive Test

$$P(+) = P(+ \mid D)P(D) + P(+ \mid \neg D)P(\neg D)$$
$$= (0.99)(0.01) + (0.05)(0.99)$$
$$= 0.0099 + 0.0495 = 0.0594$$

---

Step 2: Compute Posterior Probability

$$P(D \mid +) = \frac{0.99 \times 0.01}{0.0594} = \frac{0.0099}{0.0594} \approx 0.1667$$

---

📊 Final Answer

👉 Probability of actually having the disease ≈ 16.7%

---

Even though the test is 99% accurate, a positive result does not mean a 99% chance of disease.

Key Insight:

• The disease is rare (low prior)
• False positives accumulate among healthy people

---

🎯 Why It Matters

Bayes’ Theorem is important because it:

• Provides a systematic way to update beliefs
• Handles uncertainty effectively
• Forms the foundation of many fields, including:
  • Machine Learning
  • Statistics
  • Artificial Intelligence
  • Decision Theory

---

🧩 Key Takeaway

Bayes’ Theorem is not just a formula—it’s a framework for reasoning under uncertainty and continuously improving predictions as new data arrives.