Machine Learning PCCST503 Semester 5 KTU CS 2024 Scheme - Dr Binu V P

  📘 Supervised Machine Learning 🔹  Definition Supervised Learning is a type of Machine Learning where the model is trained using a labeled dataset , i.e., each input has a corresponding correct output(label). 📌 Formal Idea Given: Input X X Output Y Y We learn a function: f : X → Y f: X \rightarrow Y 👉 Goal: Predict output for unseen inputs accurately. How it works: The algorithm learns to map inputs to outputs It identifies patterns and relationships within the data The goal is to generalize these patterns so it can handle new, unseen data 🔹 Key Characteristics Uses labeled data Learning is guided by a teacher (labels) Objective: Minimize prediction error Widely used in real-world applications 🔹How It Works (Step-by-Step) 1. Training Phase Provide dataset: (input, output) pairs Model learns relationship During training, the model improves its performance by: Comparing its predictions with the actual labels Calculating t...

  Stages of Machine Learning: From Data Collection to Deployment Machine learning isn’t just about training a model—it’s a pipeline of interconnected steps . Each stage affects the final performance, reliability, and usefulness of the system. 1. Define the Problem & Collect Data This is the foundation. If this step is wrong, everything else suffers. 🔹 What happens here: Clearly define: What problem are you solving? (e.g., fraud detection, price prediction) What type of task? (classification, regression, clustering) Collect relevant data from: Databases APIs Sensors User interactions 🔹 Why it matters: Good models require relevant + high-quality data Poor data = poor predictions (no matter how advanced the model) 2. Data Preparation & Cleaning Raw data is messy. This stage makes it usable. 🔹 Key tasks: Handle missing values (fill or remove) Remove duplicates Detect and treat outliers Normalize/scale numerical features Conve...

  📘 Numerical Problem: Binomial + Beta (MLE & MAP) 📝 Problem A coin is tossed 10 times , and the number of heads observed is 7 . Assume: X ∼ Binomial ( n = 10 , p ) X \sim \text{Binomial}(n=10, p) Tasks: Find the MLE estimate of p p Assume prior: p ∼ Beta ( α = 2 ,    β = 2 ) p \sim \text{Beta}(\alpha = 2,\; \beta = 2) Find the MAP estimate of p p Compare the results ✅ Step 1: MLE Estimate For Binomial distribution: p ^ M L E = k n \hat{p}_{MLE} = \frac{k}{n} ​ Where: k = 7 k = 7 ,  n = 10 n = 10 p ^ M L E = 7 10 = 0.7 \hat{p}_{MLE} = \frac{7}{10} = 0.7 Var ^ = p ^ ( 1 − p ^ ) =0.7*0.3=0.21 \widehat{\text{Var}} = \hat{p}(1 - \hat{p}) ✅ Step 2: MAP Estimate 🔹 Formula (Very Important) p ^ M A P = k + α − 1 n + α + β − 2 \hat{p}_{MAP} = \frac{k + \alpha - 1}{n + \alpha + \beta - 2} ​ 🔹 Substitute values k = 7 k = 7 n = 10 n = 10 α = 2 \alpha = 2 , β = 2 \beta = 2 p ^ M A P = 7 + 2 − 1 10 + 2 + 2 − 2 = 8 12 = 0.667 \hat{p}_{MAP} ...

  📘 Maximum A Posteriori (MAP) Estimation 🔹 Definition MAP estimation is a method for estimating model parameters by maximizing the posterior probability of the parameters given the observed data. 👉 In simple terms: It finds parameters that are most probable after considering both data and prior knowledge . 🔹 Mathematical Formulation Using Bayes’ theorem: P ( θ ∣ D ) = P ( D ∣ θ ) P ( θ ) P ( D ) P(\theta\mid D)=\frac{P(D\mid \theta)P(\theta)}{P(D)} 👉 MAP estimate: θ ^ M A P = arg ⁡ max ⁡ θ P ( θ ∣ D ) \hat{\theta}_{MAP} = \arg\max_{\theta} P(\theta \mid D) 🔹Simplified Optimization Form Since P ( D ) P(D)  is constant: θ ^ M A P = arg ⁡ max ⁡ θ P ( D ∣ θ ) ⋅ P ( θ ) \hat{\theta}_{MAP} = \arg\max_{\theta} P(D \mid \theta) \cdot P(\theta) Taking log: θ ^ M A P = arg ⁡ max ⁡ θ [ log ⁡ P ( D ∣ θ ) + log ⁡ P ( θ ) ] \hat{\theta}_{MAP} = \arg\max_{\theta} \left[ \log P(D \mid \theta) + \log P(\theta) \right] 🔹  Interpretation MAP combines: Likelihood ...