Introduction to Regression in Machine Learning

 

Introduction to Regression in Machine Learning

🔹 What is Regression?

Regression is a type of supervised machine learning technique used to predict continuous (numeric) values.

👉 In simple terms:

Regression finds a relationship between input variables (features) and a continuous output.

🔹  Basic Idea

We learn a function:

y=f(x)y = f(x)

Where:

  • xx → input features
  • yy → continuous output

🔹Example

📌 House Price Prediction

  • Inputs: area, number of rooms, location
  • Output: price

👉 Regression model predicts:

price=f(features)\text{price} = f(\text{features})

🔹 Types of Regression

🟦 1. Linear Regression (Most Important)

y=wx+by = wx + b
  • Simple linear relationship
  • Easy to understand and widely used

🟩 2. Multiple Linear Regression

y=w1x1+w2x2+...+by = w_1x_1 + w_2x_2 + ... + b
  • Uses multiple features

🟨 3. Polynomial Regression

  • Captures non-linear relationships

🟩 4. Logistic Regression (Classification)

  • Used when output is categorical (0 or 1).

👉 Instead of predicting value directly, it predicts probability.

Example

  • Spam detection
  • Disease prediction

🔹How Regression Works

  1. Collect data
  2. Represent data as feature vectors
  3. Choose model (e.g., linear)
  4. Define loss function (e.g., MSE)
  5. Optimize parameters

🔹 Loss Functions

📌 Linear Regression

L=(y−y^)2L = (y - \hat{y})^2

👉 Mean Squared Error (MSE)

📌 Logistic Regression 

L=−[ylog⁡(y^)+(1−y)log⁡(1−y^)]L = -[y \log(\hat{y}) + (1-y)\log(1-\hat{y})]

👉 Called Cross-Entropy Loss


🔹Applications of Regression

🏠 1. Real Estate

  • Predict house prices

📈 2. Finance

  • Stock price prediction
  • Risk analysis

🌡️ 3. Weather Forecasting

  • Temperature prediction

🏥 4. Healthcare

  • Predict patient recovery time
  • Disease Prediction

🚗 5. Transportation

  • Predict travel time

🔹 Advantages

  • Simple and interpretable
  • Works well for many real-world problems
  • Efficient to train

🔹Limitations

  • Assumes relationship form (e.g., linear)
  • Sensitive to outliers
  • May underfit complex data

🔹 Regression vs Classification

Feature        Regression        Classification
Output            Continuous        Discrete
Example         Price        Spam/Not spam

📝 Summary 

  • Regression models relationships between input and output variables
  • Linear regression → continuous output
  • Logistic regression → classification via probability
  • Common model:
y=wx+by = wx + b
  • Uses loss functions like MSE

Comments

Post a Comment

Popular posts from this blog

Machine Learning PCCST503 Semester5 KTU CS 2024 Scheme - Dr Binu V P

About Me  - Dr Binu V P Syllabus Machine Learning PCCST503 KTU 2024 Scheme Model Question Paper Machine Learning PCCST503 Semester 5 KTU CS 2024 Scheme Do the programs to better understand the concept. Machine Learning Lab PCCSL508 Module -I Introduction to Machine Learning Introduction to ML Distinguishing Machine Learning from Traditional Programming Machine Learning Paradigms Supervised UnSupervised Semi-Supervised Reinforcement Stages of Machine Learning: From Data Collection to Deployment Parameter Estimation Introduction to Parameter Estimation in Machine Learning Maximum Likelyhood Estimation.(MLE) Maximum A Posteriori Estimation.(MAP) Numerical Problems MLE and MAP Supervised Learning Feature Representation and Problem Formulation Role of loss functions and optimization  Regression Introduction to Regression   Linear regression with one variable  (  lab experiment ) Example Problems- simple linear regression - University Questions Evaluation Metri...
Read more

Introduction to Machine Learning (ML)

  📘 Introduction to Machine Learning (ML) 🔹 What is Machine Learning? Machine Learning is a subfield of Artificial Intelligence that focuses on developing algorithms that allow computers to learn patterns from data and make decisions or predictions without explicit programming . It focuses on developing algorithms and models that allow machines to: Identify patterns in data Make decisions with minimal human intervention Predict future outcomes based on past experiences By continuously learning from new data, machine learning systems become more accurate and efficient over time. 📌 Formal Definition  A computer program is said to learn from experience E with respect to some task T and performance measure P if its performance at task T , as measured by P , improves with experience E . 🔹 Why Machine Learning? Traditional programming: Input + Program → Output Machine Learning paradigm: Input + Output → Model (learned program) 👉 Instead of writing...
Read more

Distinguishing Machine Learning from Traditional Programming

Image
  📘 Distinguishing Machine Learning from Traditional Programming 🔹 1. Fundamental Idea 🧾 Traditional Programming In conventional programming, developers write explicit instructions for the system to follow. The programmer explicitly writes rules/logic System follows deterministic instructions Input data is processed using predefined rules The output is based strictly on those rules 👉 Flow: Input + Program → Output 🤖 Machine Learning In machine learning, the system learns patterns from data instead of being explicitly programmed. Programmer provides  data + learning algorithm The model is trained using data The system  learns rules automatically from data It makes predictions or decisions based on learned patterns 👉 Flow: Input + Output → Learning Algorithm → Model Model + New Input → Predicted Output 🔹 2. Key Difference in Approach Aspect      Traditional Programming Machine Learning Input method Logic creation   ...
Read more