Naïve Bayes Classifier 

🔍 What is Naïve Bayes?

The Naïve Bayes classifier is a supervised machine learning algorithm used for classification tasks. It is based on Bayes’ Theorem and predicts the class of a data point using probabilities.

👉 It is called:

• “Bayes” → because it is based on Bayes’ theorem
• “Naïve” → because it assumes independence between features

This means:

The presence of one feature does not affect the presence of another feature.

---

🧠 Core Idea

In classification, we want to compute:

$$P(h \mid d)$$

👉 Probability of class $h$ given data $d$

Using Bayes’ theorem:

$$P(h|d) = \frac{P(d|h)\cdot P(h)}{P(d)}$$

---

🎯 Goal: Choose Best Class (MAP)

We compute probabilities for all classes and choose the maximum:

$$MAP(h) = \arg\max P(d|h)\cdot P(h)$$

👉 This is called Maximum A Posteriori (MAP)

• We ignore $P(d)$ because it is constant for all classes

---

⚙️ Why “Naïve”? (Key Assumption)

Instead of computing:

$$P(d_1, d_2, d_3 | h)$$

Naïve Bayes assumes:

$$P(d|h) = P(d_1|h)\cdot P(d_2|h)\cdot P(d_3|h)\cdots$$

👉 This simplifies computation drastically

📌 Important:

• This assumption is rarely true in real life
• But the algorithm still performs surprisingly well

---

📊 How Naïve Bayes Works (Step-by-Step)

1. 🧾 Calculate Class Probabilities (Prior)

$$P(class) = \frac{\text{count of class}}{\text{total instances}}$$

Example:

• If 50 out of 100 emails are spam:

  $$P(spam) = 0.5$$
  

---

2. 🔗 Calculate Conditional Probabilities (Likelihood)

For each feature:

$$P(feature \mid class)$$

Example:

$$P(money \mid class = spam)$$

👉 Calculated as:

$$\frac{\text{count(feature AND class)}}{\text{count(class)}}$$

---

3. 🧮 Compute Posterior for Each Class

$$P(class \mid data) \propto P(data \mid class)\cdot P(class)$$

---

4. 🏆 Choose Class with Maximum Probability

👉 Highest value = predicted class

---

📧 Example: Spam Detection

We want:

$$P(spam \mid money)$$
$$= \frac{P(money \mid spam)\cdot P(spam)}{P(money)}$$

Interpretation:

• $P(spam)$ → Prior (how common spam is)
• $P(money\mathrm{\mid }spam)\; \to \; Likelihood\; (word\; “money”\; in\; spam)$
• $P(money)$→ Evidence
• $P(spam\mathrm{\mid }money)\; \to \; Final\; probability$

👉 If:

$$P(spam|money) > P(not\_spam|money)$$

➡️ classify as spam

---

⚡ Advantages

• ✅ Simple and easy to implement
• ✅ Very fast (good for real-time prediction)
• ✅ Works well for multi-class problems
• ✅ Excellent for text classification

Examples:

• Spam filtering
• Sentiment analysis
• Document classification

---

⚠️ Disadvantages

• ❌ Assumes feature independence (not realistic)
• ❌ Cannot capture relationships between features

---

📌 Types of Naïve Bayes

1. Gaussian Naïve Bayes

• For continuous data
• Assumes normal distribution

2. Multinomial Naïve Bayes

• For text data (word counts)
• Used in NLP tasks

3. Bernoulli Naïve Bayes

• For binary features
• Example: word present or not

---

🌍 Applications

• 📧 Spam detection
• 😊 Sentiment analysis
• 🏥 Medical diagnosis
• 💳 Credit scoring
• 🎯 Recommendation systems

---

💡 Final Insight

You can summarize Naïve Bayes as:    

“Calculate probability of each class using Bayes’ theorem, assume features are independent, and pick the most probable class.”