ROC and AUC

 

📊 What is ROC?

🔹 Definition

ROC (Receiver Operating Characteristic) Curve is a graphical tool used to evaluate the performance of a binary classification model across all possible threshold values.

---

🧠 Key Idea

ROC shows how well a model can distinguish between two classes (positive vs negative).

---

📈 Axes of ROC Curve

• X-axis → False Positive Rate (FPR)
• Y-axis → True Positive Rate (TPR)

---

📐 Important Metrics

🔹 True Positive Rate (TPR) / Recall / Sensitivity

$TPR = \frac{TP}{TP + FN}$

---

🔹 False Positive Rate (FPR)

$FPR = \frac{FP}{FP + TN}$

---

🎯What Does ROC Curve Represent?

Each point on the ROC curve represents:

• A different threshold (ρ)
• Corresponding TPR and FPR values

---

🔹 Threshold Concept

• If score ≥ threshold → classify as positive
• If score < threshold → classify as negative

---

🔁 Varying Threshold

• High threshold → fewer positives → low TPR, low FPR
• Low threshold → more positives → high TPR, high FPR

---

📊 How ROC Curve is Constructed

---

🔹 Step-by-Step

1. Compute prediction scores $S(x_i)$
   
2. Sort data by score (descending)
3. Start with threshold $\rho = \infty$ → all negative
   • TPR = 0, FPR = 0 → point (0,0)
4. Gradually decrease threshold:
   • Add points to positive class
   • Update TP and FP
5. End at threshold $\rho =-\mathrm{\infty \to \; all\; positive}$
   • TPR = 1, FPR = 1 → point (1,1)

---

📍Important Points on ROC Curve

Point	Meaning
(0,0)	All predicted negative
(1,1)	All predicted positive
(0,1)	Perfect classifier ✅

---

⭐Ideal Classifier

Best point: (FPR = 0, TPR = 1)

• No false positives
• All positives detected

---

📉  Random Classifier

• ROC is a diagonal line
• Points like (0.5, 0.5), (0.9, 0.9)

👉 Represents random guessing

It is evident from the plot that the AUC for the Logistic Regression ROC curve is higher than that for the KNN ROC curve. Therefore, we can say that logistic regression did a better job of classifying the positive class in the dataset.

📊 Shape of ROC Curve

---

🔹 Good Model

• Curve bends toward top-left

🔹 Poor Model

• Curve near diagonal

Example Insight

• Logistic Regression AUC > KNN
  👉 Logistic Regression performs better

Why ROC is Useful

---

✅ Advantages

• Evaluates model across all thresholds
• Works well with imbalanced datasets
• Compares multiple models easily

📊 What is AUC?

🔹 Definition

AUC (Area Under the ROC Curve) is a single scalar value that summarizes the performance of a classification model.

It measures how well the model can separate (distinguish) positive and negative classes.

---

📐 Mathematical Meaning

• AUC is the area under the ROC curve, which lies between 0 and 1
• It represents the integral of TPR over FPR

---

🧠Core Interpretation (Most Important)

⭐ Probabilistic Meaning

AUC = Probability that:
A randomly chosen positive instance is ranked higher than a randomly chosen negative instance

✔ Example:

• AUC = 0.85
  👉 85% chance model ranks positive higher than negative

---

📈AUC as Measure of Separability

AUC represents the degree of separability

---

🔍 Meaning

• High AUC → classes are well separated
• Low AUC → classes overlap

---

📊 Intuition

Imagine two distributions:

• Positive scores → high values
• Negative scores → low values

👉 Better model → less overlap → higher AUC

---

📊 Range and Interpretation

AUC Value	Interpretation
1.0	Perfect classifier
0.9 – 0.99	Excellent
0.8 – 0.9	Good
0.7 – 0.8	Fair
0.5	Random guessing
< 0.5	Worse than random

---

🔹 Important Insight

• AUC = 0.5 → no discrimination (random model)
• AUC = 1 → perfect separation

👉 Matches your text and standard references

---

🧠  Why AUC is Powerful

---

✅ 1. Threshold Independent

• Evaluates model across all thresholds (ρ)
• No need to choose a specific cutoff

👉 This is a major advantage over accuracy

---

✅ 2. Works for Imbalanced Data

• Unlike accuracy, AUC is not biased by class distribution

---

✅ 3. Model Comparison

• Higher AUC → better model
• Used to compare classifiers

---

📊 Geometric Interpretation

---

🔹 Area Calculation

• AUC is computed using:
  • Rectangles
  • Trapezoids

👉 Total area = sum of small regions 

🎯 Ideal vs Real Models

---

🔹 Ideal Classifier

• Point: (0,1)
• AUC = 1

---

🔹 Random Classifier

• Diagonal line
• AUC = 0.5

---

🔹 Poor Classifier

• Curve below diagonal
• AUC < 0.5

👉 Can be improved by flipping predictions

---

📊Key Insight 

“A classifier with curve closer to top-left is better”

✔ This directly means:

• Higher TPR
• Lower FPR
• Higher AUC

---

🧠Intuitive Understanding

---

🎓 Medical Example

• Positive = disease
• Negative = healthy

👉 AUC answers:

“How well can the model distinguish diseased vs healthy patients?”

---

📌Key Takeaways

• AUC = area under ROC curve
• Measures classification ability across all thresholds
• Represents probability of correct ranking
• Higher AUC → better separability
• Independent of threshold
• Robust for imbalanced datasets

---

🎯 Final Insight 

AUC answers the fundamental question:

👉 “How well does the model rank positive instances higher than negative ones?

📊 ROC / AUC Algorithm

The area under the ROC curve, abbreviated AUC, can be used as a measure of classifier performance. Because the total area of the plot is 1, the AUC lies in the interval [0,1] – the higher the better. The AUC value is essentially the probability that the classifier will rank a random positive test case higher than a random negative test instance.

---

🧠 Overview

The algorithm describes the steps for:

• Plotting the ROC curve
• Computing the Area Under the Curve (AUC)

---

🔹 Inputs

• Testing dataset: $D$
  
• Classifier: $M$
  

---

🔹 Step 1: Compute Scores

For each test point $x_i \in D$:

• Compute the score for the positive class $(c_1)$:

$$S(x_i)$$

---

🔹 Step 2: Sort Data

• Form pairs $(S(x_i), y_i)$
  
• Sort them in decreasing order of scores

---

🔹 Step 3: Initialize Threshold

• Set threshold:

$$\rho = \infty$$

• Initially:
  • All points are classified as negative
  • $TP = 0$, $FP = 0$
    

👉 Corresponds to point (0,0) in ROC space

---

🔹 Step 4: Iterate Over Sorted Data

For each distinct score value:

• Set threshold:

$$\rho = S(x_i)$$

• Determine predicted positive set:

$$R_1(\rho) = \{ x_i \in D : S(x_i) > \rho \}$$

---

🔹 Step 5: Update Counts

For each test point:

• If $y_i = c_1$(positive class):
  • Increment True Positives (TP)
• Else:
  • Increment False Positives (FP)

---

🔹 Step 6: Compute ROC Point

For each threshold, compute:

• False Positive Rate (FPR)
• True Positive Rate (TPR)

$$(FPR, TPR) = \left( \frac{FP}{n_2}, \frac{TP}{n_1} \right)$$

where:

• $n_1$ = number of positive instances
• $n_2$ = number of negative instances

---

🔹 Step 7: Final Point

• At the end:
  • All points are classified as positive
  • $FN = 0$, $TN = 0$
    

👉 Corresponds to point (1,1) in ROC space

---

📈 AUC Computation

---

🔹 Idea

AUC is computed incrementally as new points are added to the ROC curve.

---

🔹 Maintain Previous Values

• Previous values:
  • $FP_{prev}$
  • $TP_{prev}$

---

🔹 Define Points

$$(x_1, y_1) = \left( \frac{FP_{prev}}{n_2}, \frac{TP_{prev}}{n_1} \right)$$
$$(x_2, y_2) = \left( \frac{FP}{n_2}, \frac{TP}{n_1} \right)$$
$$(x_1, 0) = \left( \frac{FP_{prev}}{n_2}, 0 \right)$$
$$(x_2, 0) = \left( \frac{FP}{n_2}, 0 \right)$$

---

🔹 Area Calculation

These four points define:

• A trapezoid if $x_2 > x_1$ and $y_2 > y_1$
• Otherwise, a rectangle (possibly degenerate)

---

🔹 Trapezoid Area Formula

$\text{Area} = b \cdot h, \quad b = |x_2 - x_1|, \quad h = \frac{1}{2}(y_2 + y_1)$

---

🔹 Interpretation

• Total AUC = sum of areas of all trapezoids/rectangles
• Represents total area under ROC curve

---

🎯 Key Insight

The algorithm constructs the ROC curve point by point and computes AUC as the sum of incremental areas.

📊 Example

Consider the following sorted scores, along with the true class, for some testing dataset:

• $n = 5$
• $n_1 = 3$ (number of positive instances)
• $n_2 = 2$ (number of negative instances)

---

🔹 Data (Score, Class)

$$(0.9, c_1),\ (0.8, c_2),\ (0.8, c_1),\ (0.8, c_1),\ (0.1, c_2)$$

Figure shows the ROC plot, with the shaded region representing the AUC.

We can observe that a trapezoid is obtained whenever there is at least one positive and one negative point with the same score.

The total AUC is 0.833, obtained as the sum of:

• the trapezoidal region on the left (0.333)
• and the rectangular region on the right (0.5)

📈 Types of Classifiers on ROC Curve

🔹 1. Random Classifier

• Represented by a diagonal line
• Has no predictive power
• Performs like random guessing

---

🔹 2. Perfect Classifier

• Located at the top-left corner (0,1)
• TPR = 1, FPR = 0
• Makes no errors

---

🔹 3. Real-World Classifiers

• Fall between random and perfect
• Provide better-than-random predictions
• Goal: stay close to top-left corner

---

🎯 Ideal Behavior

A good classifier should:

• Have high True Positive Rate (TPR)
• Have low False Positive Rate (FPR)
• “Hug” the upper-left corner of the ROC plot

📊 Example Question

Suppose there are three classifiers A, B, and C.
The $(FPR,TPR)\; values\; for\; the\; classifiers\; are:$

• A: $(0, 1)$
  
• B: $(1, 1)$
  
• C: $(1, 0.5)$
  

👉 Which classifier can be considered a perfect classifier? Justify your answer.

---

✅ Answer

✔️ Classifier A (0, 1) is the perfect classifier.

---

🧠 Justification

A perfect classifier has:

• False Positive Rate (FPR) = 0 → No false positives
• True Positive Rate (TPR) = 1 → All positives correctly identified

---

📊 Evaluation of Each Classifier

🔹 Classifier A: (0, 1)

• FPR = 0 → No false positives
• TPR = 1 → All positives detected

✅ Perfect classification

---

🔹 Classifier B: (1, 1)

• FPR = 1 → All negatives misclassified
• TPR = 1 → All positives detected

❌ Not perfect (too many false positives)

---

🔹 Classifier C: (1, 0.5)

• FPR = 1 → All negatives misclassified
• TPR = 0.5 → Only half positives detected

❌ Poor classifier

---

🎯 Final Conclusion

Classifier A is perfect because it achieves:

• Zero false positives
• 100% true positive rate

Example

📊 AUC Area Calculation

---

🔹 Step 1: Expression

$\text{Area} = (0.4 - 0) \cdot \frac{1}{2}(0.6 + 0.6) + (0.6 - 0.4) \cdot \frac{1}{2}(0.8 + 0.8) + (1.0 - 0.6) \cdot \frac{1}{2}(1 + 1)$

🔹 Step 2: Simplification

$\text{Area} = 0.4 \cdot 0.6 + 0.2 \cdot 0.8 + 0.4 \cdot 1$

---

🔹 Step 3: Final Calculation

$\text{Area} = 0.24 + 0.16 + 0.4 = 0.8$

---

🎯 Final Answer

👉 AUC = 0.8

📊 Example

Given the following data, construct the ROC curve of the data. Compute the AUC. 

---

🔹 Data Table

Threshold	TP	TN	FP	FN
1	0	25	0	29
2	7	25	0	22
3	18	24	1	11
4	26	20	5	3
5	29	11	14	0
6	29	0	25	0
7	29	0	25	0

---

🔹 Python Code for Plotting ROC and Computing AUC

import matplotlib.pyplot as plt
import numpy as np

TP = np.array([0,7,18,26,29,29,29])
TN = np.array([25,25,24,20,11,0,0])
FP = np.array([0,0,1,5,14,25,25])
FN = np.array([29,22,11,3,0,0,0])

TPR = TP/(TP+FN)
FPR = FP/(FP+TN)

plt.plot(FPR,TPR)
plt.scatter(FPR,TPR)

x = zip(FPR,TPR)
for i in x:
    print(i)

area = 0
for i in range(6):
    area = area + (FPR[i+1]-FPR[i])*((TPR[i]+TPR[i+1])/2)

print("Area under the curve=", area)

---

🔹 Output

(0.0, 0.0)
(0.0, 0.2413793103448276)
(0.04, 0.6206896551724138)
(0.2, 0.896551724137931)
(0.56, 1.0)
(1.0, 1.0)
(1.0, 1.0)

Area under the curve = 0.919999

---

🎯 Final Result

👉 AUC ≈ 0.92