Evaluation Metrics for Linear Regression

 

Evaluation Metrics for  Linear Regression

πŸ‘‰ In regression, we measure how close predicted values y^\hat{y} are to actual values yy.

πŸ”Ή 1. Mean Squared Error (MSE) 

MSE=1n∑(yi−y^i)2\text{MSE} = \frac{1}{n} \sum (y_i - \hat{y}_i)^2

πŸ“Œ Interpretation:

  • Penalizes large errors heavily
  • Lower MSE → better model

πŸ”Ή 2. Root Mean Squared Error (RMSE)

RMSE=1n∑(yi−y^i)2\text{RMSE} = \sqrt{\frac{1}{n} \sum (y_i - \hat{y}_i)^2}

πŸ“Œ Interpretation:

  • Same units as yy
  • Easier to understand than MSE

πŸ”Ή 3. Mean Absolute Error (MAE)

MAE=1n∑∣yi−y^i∣\text{MAE} = \frac{1}{n} \sum |y_i - \hat{y}_i|

πŸ“Œ Interpretation:

  • Measures average error
  • Less sensitive to outliers

πŸ”Ή 4. R-Squared (R^2) -Coefficient of Determination

πŸ”Ή Definition

The R-squared (R2R^2) value measures how well the regression model explains the variability of the data.

πŸ”Ή Formula

R2=1−SSRSSTR^2 = 1 - \frac{\text{SSR}}{\text{SST}}

Where:

πŸ“Œ 1. SSR (Sum of Squared Residuals)

SSR=∑(yi−y^i)2\text{SSR} = \sum (y_i - \hat{y}_i)^2
  • Also called Sum of Squared Errors (SSE)
  • Measures unexplained variation
  • Error between actual and predicted values

πŸ“Œ 2. SST (Total Sum of Squares)

SST=∑(yi−yΛ‰)2\text{SST} = \sum (y_i - \bar{y})^2
  • Measures total variation in data
  • Deviation of actual values from mean
​

πŸ”Ή Interpretation

  • Measures goodness of fit
  • Range: 0≤R2≤10 \le R^2 \le 1
R2R^2
Meaning
R2=1R^2 = 1
    Perfect fit
R2=0    Model explains nothing
0<R2<10 < R^2 < 1
    Partial explanation

πŸ”Ή Key Insight

R2=Explained VariationTotal VariationR^2 = \frac{\text{Explained Variation}}{\text{Total Variation}}

πŸ‘‰ It tells:

“How much of the variation in y is explained by the model”​


πŸ”Ή 5. Adjusted R-Squared

Adjusted R2=1−((1−R2)(n−1)n−p−1)\text{Adjusted } R^2 = 1 - \left( \frac{(1 - R^2)(n-1)}{n - p - 1} \right)

Where:

  • pp = number of predictors

πŸ‘‰ Useful when comparing models

πŸ”Ή 6. Residual Sum of Squares (RSS)

RSS=∑(yi−y^i)2\text{RSS} = \sum (y_i - \hat{y}_i)^2

πŸ‘‰ Total squared error

πŸ”Ή 7. Total Sum of Squares (TSS)

TSS=∑(yi−yΛ‰)2\text{TSS} = \sum (y_i - \bar{y})^2

πŸ‘‰ Total variation in data

πŸ”Ή 8. Explained Sum of Squares (ESS)

ESS=∑(y^i−yΛ‰)2\text{ESS} = \sum (\hat{y}_i - \bar{y})^2

πŸ‘‰ Variation explained by model

πŸ“Œ Important Relation:

TSS=ESS+RSS\text{TSS} = \text{ESS} + \text{RSS}

πŸ”Ή 9. Mean Absolute Percentage Error (MAPE)

MAPE=100n∑∣yi−y^iyi∣\text{MAPE} = \frac{100}{n} \sum \left| \frac{y_i - \hat{y}_i}{y_i} \right|

πŸ“Œ Interpretation:

  • Error in percentage
  • Useful in business applications

πŸ”Ή 10. Residual Analysis (Conceptual)

πŸ‘‰ Residual:

ei=yi−y^ie_i = y_i - \hat{y}_i

Used to check:

  • Linearity
  • Constant variance
  • Model assumptions

πŸ”Ή 11. Summary Table

MetricFormulaUse
MSE        squared error        Penalizes large errors
RMSE        sqrt(MSE)        Interpretable
MAE        absolute error        Robust
R2R^2
        goodness of fit        Explained variance
RSS        total error        Model fit
MAPE        % error        Business

Key Insights (From ML Perspective)

  • Loss function (MSE) → used during training
  • Metrics → used for evaluation
  • MSE and R2are important metrics
  • Error → difference between actual & predicted
  • Good model → low error, high R2R^2

Comments

Post a Comment

Popular posts from this blog

Machine Learning PCCST503 Semester5 KTU CS 2024 Scheme - Dr Binu V P

About Me  - Dr Binu V P Syllabus Machine Learning PCCST503 KTU 2024 Scheme Model Question Paper Machine Learning PCCST503 Semester 5 KTU CS 2024 Scheme Do the programs to better understand the concept. Machine Learning Lab PCCSL508 Module -I Introduction to Machine Learning Introduction to ML Distinguishing Machine Learning from Traditional Programming Machine Learning Paradigms Supervised UnSupervised Semi-Supervised Reinforcement Stages of Machine Learning: From Data Collection to Deployment Parameter Estimation Introduction to Parameter Estimation in Machine Learning Maximum Likelyhood Estimation.(MLE) Maximum A Posteriori Estimation.(MAP) Numerical Problems MLE and MAP Supervised Learning Feature Representation and Problem Formulation Role of loss functions and optimization  Regression Introduction to Regression   Linear regression with one variable  (  lab experiment ) Example Problems- simple linear regression - University Questions Evaluation Metri...
Read more

Introduction to Machine Learning (ML)

  πŸ“˜ Introduction to Machine Learning (ML) πŸ”Ή What is Machine Learning? Machine Learning is a subfield of Artificial Intelligence that focuses on developing algorithms that allow computers to learn patterns from data and make decisions or predictions without explicit programming . It focuses on developing algorithms and models that allow machines to: Identify patterns in data Make decisions with minimal human intervention Predict future outcomes based on past experiences By continuously learning from new data, machine learning systems become more accurate and efficient over time. πŸ“Œ Formal Definition  A computer program is said to learn from experience E with respect to some task T and performance measure P if its performance at task T , as measured by P , improves with experience E . πŸ”Ή Why Machine Learning? Traditional programming: Input + Program → Output Machine Learning paradigm: Input + Output → Model (learned program) πŸ‘‰ Instead of writing...
Read more

Distinguishing Machine Learning from Traditional Programming

Image
  πŸ“˜ Distinguishing Machine Learning from Traditional Programming πŸ”Ή 1. Fundamental Idea 🧾 Traditional Programming In conventional programming, developers write explicit instructions for the system to follow. The programmer explicitly writes rules/logic System follows deterministic instructions Input data is processed using predefined rules The output is based strictly on those rules πŸ‘‰ Flow: Input + Program → Output πŸ€– Machine Learning In machine learning, the system learns patterns from data instead of being explicitly programmed. Programmer provides  data + learning algorithm The model is trained using data The system  learns rules automatically from data It makes predictions or decisions based on learned patterns πŸ‘‰ Flow: Input + Output → Learning Algorithm → Model Model + New Input → Predicted Output πŸ”Ή 2. Key Difference in Approach Aspect      Traditional Programming Machine Learning Input method Logic creation   ...
Read more