Numerical Problems MLE and MAP

 

📘 Numerical Problem: Binomial + Beta (MLE & MAP)

📝 Problem

A coin is tossed 10 times, and the number of heads observed is 7.

Assume:

• $X \sim \text{Binomial}(n=10, p)$
  

Tasks:

1. Find the MLE estimate of $p$
   
2. Assume prior: $p \sim \text{Beta}(\alpha = 2,\; \beta = 2)$
   Find the MAP estimate of $p$
   
3. Compare the results

---

✅ Step 1: MLE Estimate

For Binomial distribution:

$$\hat{p}_{MLE} = \frac{k}{n}$$

Where:

• $k = 7$, $n = 10$
  

$$\hat{p}_{MLE} = \frac{7}{10} = 0.7$$

$$\widehat{\text{Var}} = \hat{p}(1 - \hat{p})$$

---

✅ Step 2: MAP Estimate

🔹 Formula (Very Important)

$$\hat{p}_{MAP} = \frac{k + \alpha - 1}{n + \alpha + \beta - 2}$$

---

🔹 Substitute values

• $k = 7$
  
• $n = 10$
  
• $\alpha = 2$, $\beta = 2$
  

$$\hat{p}_{MAP} = \frac{7 + 2 - 1}{10 + 2 + 2 - 2} = \frac{8}{12} = 0.667$$

---

✅ Final Answers

Method	Estimate
MLE	0.7
MAP	0.667

---

🔥 Key Insight

MAP = MLE + Prior influence

---

📌 Modified Problem (Stronger Prior Effect)

Suppose:

• $\alpha = 10,\; \beta = 10$
  

---

🔹 Recompute MAP

$$\hat{p}_{MAP} = \frac{7 + 10 - 1}{10 + 10 + 10 - 2} = \frac{16}{28} \approx 0.571$$

---

🔥 Comparison

Method	Value
MLE	0.7
MAP (weak prior)	0.667
MAP (strong prior)	0.571

---

🔹 Insight

• Strong prior → more influence
• Estimate moves closer to 0.5

---

📝 Summary (Exam Ready)

• MLE:

$$\hat{p} = \frac{k}{n}$$

• MAP:

$$\hat{p} = \frac{k + \alpha - 1}{n + \alpha + \beta - 2}$$

---

✅ Practice Question (Try Yourself)

A machine produces defective items. Out of 20 items, 6 are defective.

• Find MLE of defect probability
• Assume prior $\text{Beta}(3,3)$, find MAP

📘 Numerical Problem: MLE vs MAP (Gaussian Mean)

📝 Problem

Given data:

$$x = \{2,\;4,\;6,\;8\}$$

Assume:

• Data follows $\mathcal{N}(\mu, \sigma^2)$
  
• Variance $\sigma^2 = 4$ (known)

Also given prior:

• $\mu \sim \mathcal{N}({\mu }_0=5,{\tau }^2=1)$

---

🔹 Tasks

1. Find MLE estimate of $\mu$
   
2. Find MAP estimate of $\mu$
   
3. Compare both results

---

✅ Step 1: Compute Sample Mean (MLE)

$$\hat{\mu}_{MLE} = \frac{1}{n}\sum x_i$$ $$= \frac{2+4+6+8}{4} = \frac{20}{4} = 5$$

👉 MLE = 5

---

✅ Step 2: Compute MAP Estimate

Formula:

$$\hat{\mu}_{MAP} = \frac{n\bar{x} + \lambda \mu_0}{n + \lambda}$$

where:

$$\lambda = \frac{\sigma^2}{\tau^2}$$

---

🔹 Calculate $\lambda$

$$\lambda = \frac{4}{1} = 4$$

---

🔹 Substitute values

• $n = 4$
  
• $\bar{x} = 5$
  
• $\mu_0 = 5$
  

$$\hat{\mu}_{MAP} = \frac{4 \cdot 5 + 4 \cdot 5}{4 + 4} = \frac{20 + 20}{8} = \frac{40}{8} = 5$$

---

✅ Final Answers

Method	Estimate
MLE	5
MAP	5

---

🔹  Interpretation

👉 Both are same because:

• Prior mean = data mean
• No conflict between data and prior

---

📌 Modified Problem (More Interesting)

Let’s slightly change prior:

• $\mu_0 = 0$ (strong prior away from data)

---

🔹 Recompute MAP

$$\hat{\mu}_{MAP} = \frac{4 \cdot 5 + 4 \cdot 0}{8} = \frac{20}{8} = 2.5$$

---

🔥 Comparison

Method	Value
MLE	5
MAP	2.5

---

🔹 Insight

• MLE depends only on data
• MAP is pulled toward prior

👉 This is called shrinkage effect

---

📝 Summary (Exam Ready)

• MLE:

$$\hat{\mu} = \bar{x}$$

• MAP:

$$\hat{\mu} = \frac{n\bar{x} + \lambda \mu_0}{n + \lambda}$$

---

✅ Practice Question (Try Yourself)

Given:

• Data: $3,5,7$
• ${\sigma }^2=1$, ${\tau }^2=1$, ${\mu }_0=0$
  

👉 Find MLE and MAP