Semi-Supervised Learning (SSL)

 

📘 Semi-Supervised Learning (SSL)

🔹 Definition

Semi-Supervised Learning is a machine learning paradigm that uses:

• Small amount of labeled data
• Large amount of unlabeled data

👉 It combines ideas from:

• Supervised Learning (with labels)
• Unsupervised Learning (without labels)

SSL aims to learn a function that predicts outputs using limited labeled data and abundant unlabeled data.

Example of Semi-Supervised Learning

Scenario:

Imagine you’re building a model to classify images of animals such as:

• Dogs
• Cats
• Birds

Labeling thousands of images manually would require significant time and effort. However, you have access to millions of unlabeled images.

---

Approach:

With semi-supervised learning:

• You start with a small set of labeled images to train the model
• The model then predicts labels for the large set of unlabeled data
• These predictions help the model learn better patterns and improve accuracy

---

Outcome:

➡️ The model improves its performance without needing a fully labeled dataset

---

Applications:

This approach is commonly used in:

• Speech recognition
• Image classification
• Text categorization

---

Key Idea:

➡️ Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data to build efficient models.

---

🔹 Intuition (Easy Understanding)

A commonly used analogy:

• Supervised learning → Student learns with a teacher
• Unsupervised learning → Student learns alone
• Semi-supervised learning → Teacher gives some guidance + student practices independently

👉 This combination improves learning efficiency.

---

🔹Why Semi-Supervised Learning?

📌 Key Motivation

• Labeling data is:
  • Expensive
  • Time-consuming
  • Requires experts
• Unlabeled data is:
  • Easily available
  • Large in volume

👉 SSL bridges this gap.

📌 When to Use 

SSL is used when:

• Labeled data is scarce
• Unlabeled data is abundant
• Data labeling is costly (e.g., medical images, speech data)

---

🔹Mathematical Formulation

Let:

• $L = \{(x_i, y_i)\}$ → labeled dataset
• $U = \{x_j\}$ → unlabeled dataset

👉 Goal:
Learn a function:

$$f: X \rightarrow Y$$

using both labeled and unlabeled data.

---

🔹 Working of Semi-Supervised Learning

📌 General Process

1. Train model using labeled data
2. Predict labels for unlabeled data
3. Select high-confidence predictions
4. Add them to labeled dataset
5. Retrain model
6. Repeat

👉 This iterative improvement is key.

---

🔹 Important Techniques

🟦 1. Self-Training

• Model trains on labeled data
• Predicts labels for unlabeled data
• Adds high-confidence predictions

📌 Known as pseudo-labeling

👉 Risk: Error propagation

---

🟩 2. Co-Training

• Two models trained on different feature sets
• Each model labels data for the other

👉 Uses multiple “views” of data

---

🟨 3. Multi-View Learning

• Extension of co-training
• Uses different representations (e.g., image + text)

---

🟥 4. Graph-Based Methods

📌 Idea:

• Represent data as a graph:
  • Nodes → data points
  • Edges → similarity

👉 Labels propagate from labeled to unlabeled nodes

📌 Key concept:

• “Labels spread through connections in the graph”

---

🔹Real-World Examples

📸 1. Image Classification

• Few labeled images
• Thousands of unlabeled images

👉 Used in:

• Face recognition
• Object detection

---

📝 2. Text Classification

• Few labeled documents
• Large unlabeled corpus

👉 Applications:

• Spam filtering
• News classification

---

🎤 3. Speech Recognition

• Limited labeled audio
• Large unlabeled recordings

---

💳 4. Fraud Detection

• Detect anomalies in financial transactions

👉 Used by companies like PayPal

---

🔐 5. Security Systems

• Malware detection
• Network anomaly detection

---

🔹 Advantages

✅ 1. Cost Efficient

• Reduces need for manual labeling

✅ 2. Better Generalization

• Uses more data → better performance

✅ 3. Handles Real-world Data

• Works well with large unstructured datasets

✅ 4. Improves Rare Class Detection

• Helps when labeled samples are few

---

🔹Limitations

❌ 1. Error Propagation

• Wrong labels can degrade performance

❌ 2. Assumption Sensitivity

• Depends on cluster/smoothness assumptions

❌ 3. Complex Models

• Requires careful tuning

❌ 4. Evaluation Difficulty

• Hard to measure performance with limited labels

---

🔹Key Assumptions (Very Important)

1. Smoothness Assumption
   → Nearby points have same label
2. Cluster Assumption
   → Data forms clusters
3. Manifold Assumption
   → Data lies in lower-dimensional space

---

🔹Comparison with Other Learning Types

Feature	Supervised	Unsupervised	Semi-Supervised
Labeled Data	Required	Not used	Limited
Unlabeled Data	Not used	Used	Used
Accuracy	High (if enough labels)	Lower	High with fewer labels

---

📝 Summary 

• Semi-supervised learning = small labeled + large unlabeled data
• Combines supervised and unsupervised learning
• Uses techniques like:
  • Self-training
  • Co-training
  • Graph-based methods
• Applications:
  • Vision, NLP, healthcare, finance
• Key benefit:
  • Reduces labeling cost while improving accuracy