Naive Bayes Classifier - Example Problems

 

Example1: Spam Detection using Naïve Bayes

🧾 Step 1: Training Dataset

Email ID	Contains “offer”	Contains “money”	Class
1	Yes	Yes	Spam
2	Yes	No	Spam
3	No	Yes	Spam
4	Yes	Yes	Spam
5	No	No	Not Spam
6	No	Yes	Not Spam
7	Yes	No	Not Spam
8	No	No	Not Spam

---

📊 Step 2: Compute Prior Probabilities

Total emails = 8

• Spam = 4
• Not Spam = 4

$$P(Spam) = \frac{4}{8} = 0.5$$
$$P(Not\ Spam) = \frac{4}{8} = 0.5$$

---

🔗 Step 3: Compute Likelihoods

For Spam

• $P(offer=Yes\mathrm{\mid }Spam)=3\mathrm{/}4=0.75$
• $P(money=Yes\mathrm{\mid }Spam)=3\mathrm{/}4=0.75$

---

For Not Spam

• $P(offer=Yes\mathrm{\mid }NotSpam)=1\mathrm{/}4=0.25$
• $P(money=Yes\mathrm{\mid }NotSpam)=1\mathrm{/}4=0.25$

---

🎯 Step 4: New Email Prediction

👉 New email contains:

• “offer” = Yes
• “money” = Yes

We classify this email.

---

🔢 Apply Naïve Bayes

Formula

$P(h\mid d)\propto P(d_1\mid h)\cdot P(d_2\mid h)\cdot P(h)$

---

🧮 Compute for Spam

$$P(Spam\mathrm{\mid }data)\propto 0.75\times 0.75\times 0.5=0.28125$$

---

🧮 Compute for Not Spam

$$P(NotSpam\mathrm{\mid }data)\propto 0.25\times 0.25\times 0.5=0.03125$$

---

🏆 Final Decision

• Spam score = 0.28125
• Not Spam score = 0.03125

👉 Prediction: SPAM

Example2: Problem: Should We Play?

We want to predict:

If Weather = Sunny, should we Play = Yes or No?

📊 Step 1: Sample Dataset

Day	Weather	Play
1	Sunny	No
2	Sunny	No
3	Overcast	Yes
4	Rainy	Yes
5	Rainy	Yes
6	Rainy	No
7	Overcast	Yes
8	Sunny	No
9	Sunny	Yes
10	Rainy	Yes
11	Sunny	Yes
12	Overcast	Yes
13	Overcast	Yes
14	Rainy	No

---

📈 Step 2: Frequency Table

🎯 Class Counts

• Yes = 9
• No = 5
• Total = 14

---

🌤️ Weather vs Play

For Play = Yes (9)

Weather	Count
Sunny	2
Overcast	4
Rainy	3

---

For Play = No (5)

Weather	Count
Sunny	3
Overcast	0
Rainy	2

---

📊 Step 3: Likelihood Probabilities

For Play = Yes

$$P(Sunny|Yes) = \frac{2}{9}$$

For Play = No

$$P(Sunny|No) = \frac{3}{5}$$

---

📊 Step 4: Prior Probabilities

$$P(Yes) = \frac{9}{14}, \quad P(No) = \frac{5}{14}$$

---

🔢 Step 5: Apply Bayes’ Theorem

$P(h\mid d)=\frac{P(d\mid h)\cdot P(h)}{P(d)}$

Since $P(d)\; is\; same\; for\; both\; classes,\; we\; compare:$

---

🧮 Compute for Play = Yes

$$P(Yes|Sunny) \propto \frac{2}{9} \times \frac{9}{14} = \frac{2}{14} = 0.1429$$

---

🧮 Compute for Play = No

$$P(No|Sunny) \propto \frac{3}{5} \times \frac{5}{14} = \frac{3}{14} = 0.2143$$

---

🏆 Final Decision

• Yes → 0.1429
• No → 0.2143

👉 Prediction: NO (Do not play) 

🚗 Problem: Accident Prediction Using Naïve Bayes

We want to predict:

Will an accident occur (Yes/No) based on:

• Weather Condition
• Road Condition
• Traffic Condition
• Engine Problem

---

📊 Step 1: Sample Dataset

ID	Weather	Road	Traffic	Engine	Accident
1	Sunny	Good	Low	No	No
2	Rainy	Poor	High	Yes	Yes
3	Foggy	Poor	High	No	Yes
4	Sunny	Good	Medium	No	No
5	Rainy	Poor	High	Yes	Yes
6	Sunny	Poor	Medium	Yes	Yes
7	Foggy	Good	Low	No	No
8	Rainy	Good	Medium	No	No
9	Sunny	Poor	High	Yes	Yes
10	Foggy	Poor	High	Yes	Yes

---

🎯 Problem Statement for Students

👉 Given a new condition:

• Weather = Rainy
• Road = Poor
• Traffic = High
• Engine = Yes

Question:

Will an accident occur?

---

📈 Step 2: Frequency (Class Counts)

• Accident = Yes → 6
• Accident = No → 4

---

📊 Step 3: Prior Probabilities

$$P(Yes) = \frac{6}{10} = 0.6$$
$$P(No) = \frac{4}{10} = 0.4$$

---

🔗 Step 4: Likelihood Probabilities

For Accident = Yes (6 cases)

• $P(Rainy|Yes) = 2/6$
  
• $P(Poor|Yes) = 5/6$
  
• $P(High|Yes) = 5/6$
  
• $P(Engine=Yes|Yes) = 5/6$
  

---

For Accident = No (4 cases)

• $P(Rainy\mathrm{\mid }No)=1\mathrm{/}4$
• $P(Poor|No) = 0/4 = 0$
  
• $P(High|No) = 0/4 = 0$
  
• $P(Engine=Yes|No) = 0/4 = 0$
  

---

🔢 Step 5: Apply Naïve Bayes

Formula:

$P(h\mid d)\propto P(d_1\mid h)P(d_2\mid h)P(d_3\mid h)P(d_4\mid h)\mathrm{<br>}$

---

🧮 Without Smoothing (for understanding)

For Yes

$$P(Yes|data) \propto \frac{2}{6} \times \frac{5}{6} \times \frac{5}{6} \times \frac{5}{6} \times 0.6$$

👉 Non-zero → valid

---

For No

$$P(No|data) \propto \frac{1}{4} \times 0 \times 0 \times 0 \times 0.4 = 0$$

---

🏆 Final Decision

👉 Since:

• Yes → non-zero probability
• No → 0

➡️ Prediction: Accident = YES

---

🎓 Problem: Student Performance Prediction

We want to predict:

Will a student Pass or Fail?

Based on:

• Study Hours (High / Low)
• Attendance (Good / Poor)
• Assignment Submission (Yes / No)

---

📊 Step 1: Sample Dataset

ID	Study	Attendance	Assignment	Result
1	High	Good	Yes	Pass
2	High	Good	Yes	Pass
3	High	Poor	Yes	Pass
4	Low	Good	Yes	Pass
5	Low	Poor	No	Fail
6	Low	Poor	No	Fail
7	High	Good	No	Pass
8	Low	Good	No	Fail
9	High	Poor	No	Fail
10	Low	Good	Yes	Pass

---

🎯 Problem Statement

👉 Given a new student:

• Study = High
• Attendance = Good
• Assignment = No

Question:

Will the student Pass or Fail?

---

📈 Step 2: Class Counts

• Pass = 6
• Fail = 4
• Total = 10

---

📊 Step 3: Prior Probabilities

$$P(Pass) = \frac{6}{10} = 0.6$$
$$P(Fail) = \frac{4}{10} = 0.4$$

---

🔗 Step 4: Likelihood Probabilities

For Pass (6 cases)

• $P(High|Pass) = 4/6$
  
• $P(Good|Pass) = 5/6$
  
• $P(No\ Assignment|Pass) = 1/6$
  

---

For Fail (4 cases)

• $P(High|Fail) = 1/4$
  
• $P(Good|Fail) = 1/4$
  
• $P(No\ Assignment|Fail) = 4/4 = 1$
  

---

🔢 Step 5: Apply Naïve Bayes

Formula:

$P(h\mid d) \propto P(d_1\mid h)\cdot P(d_2\mid h)\cdot P(d_3\mid h)\cdot P(h)$

---

🧮 Compute for Pass

$$P(Pass|data) \propto \frac{4}{6} \times \frac{5}{6} \times \frac{1}{6} \times 0.6$$
$$= 0.6667 \times 0.8333 \times 0.1667 \times 0.6 \approx 0.0556$$

---

🧮 Compute for Fail

$$P(Fail|data) \propto \frac{1}{4} \times \frac{1}{4} \times 1 \times 0.4$$
$$= 0.25 \times 0.25 \times 1 \times 0.4 = 0.025$$

---

🏆 Final Decision

• Pass → 0.0556
• Fail → 0.025

👉 Prediction: PASS