XOR problem- how it is solved using multi layer perceptron

 

🔹 What is the XOR Problem?

1. 🧠 Definition

The XOR (Exclusive OR) function outputs:

  • 1 when inputs are different
  • 0 when inputs are same

Truth Table:

x₁    x₂    XOR Output
0    0    0
0    1    1
1    0    1
1    1    0

2. ❌ Why is XOR a Problem for Perceptron?

A single-layer perceptron can only learn linear decision boundaries (straight lines).

XOR is not linearly separable:

  • No single straight line can separate:
    • (0,1) and (1,0) → class 1
    • (0,0) and (1,1) → class 0

➡️ This limitation is called the XOR problem.



3. 📐 Geometric Intuition

  • Points lie diagonally:
    • Class 1 → opposite corners
    • Class 0 → opposite corners

➡️ Requires non-linear separation



🔹 Solution: Multi-Layer Perceptron (MLP)

4. 🧩 Key Idea

Add a hidden layer to:

  • Transform input space
  • Create non-linear decision boundaries

5. 🏗️ Network Architecture

A simple MLP for XOR:

  • Input layer: 2 neurons (x₁, x₂)
  • Hidden layer: 2 neurons
  • Output layer: 1 neuron

6. ⚙️ How It Works

Step 1: Hidden Layer creates intermediate features

Each hidden neuron acts like a linear classifier.

Example:

  • Hidden neuron 1 → detects (x₁ OR x₂)
  • Hidden neuron 2 → detects (x₁ AND x₂)

Step 2: Output Layer combines them

Final output:

XOR=(OR)AND (NAND)\text{XOR} = (\text{OR}) - (\text{AND})XOR=(x1​∨x2​)∧¬(x1​∧x2​)

➡️ Produces correct XOR behavior.

7. 🔢 Mathematical View

Hidden layer:

h1=step(x1+x2−0.5)(OR)h_1 = \text{step}(x_1 + x_2 - 0.5) \quad (\text{OR})
h2=step(-x1-x2+1.5)(NAND)h_2 = \text{step}(x_1 + x_2 - 1.5) \quad (\text{AND})

Output layer:

y=step(h1+h2−1.5) ( AND )y = \text{step}(h_1 - h_2 - 0.5)







8. 🎯 Key Insight

The hidden layer transforms the data into a space where it becomes linearly separable.

9. 🧠 Why This Matters

  • Solving XOR showed that:
    • Single-layer networks are limited
    • Multi-layer networks are powerful

➡️ This led to the development of multi layer neural netwok and deep learning

🔹 One-Line Summary

The XOR problem shows that single-layer perceptrons cannot model non-linear patterns, but multi-layer perceptrons solve it by introducing hidden layers that enable non-linear decision boundaries.

Comments

Post a Comment

Popular posts from this blog

Machine Learning PCCST503 Semester5 KTU CS 2024 Scheme - Dr Binu V P

About Me  - Dr Binu V P Syllabus Model Question Paper Machine Learning PCCST503 Semester 5 KTU CS 2024 Scheme Do the programs to better understand the concept. Machine Learning Lab PCCSL508 Module -I Introduction to Machine Learning Introduction to ML Distinguishing Machine Learning from Traditional Programming Machine Learning Paradigms Supervised UnSupervised Semi-Supervised Reinforcement Stages of Machine Learning: From Data Collection to Deployment Parameter Estimation Introduction to Parameter Estimation in Machine Learning Maximum Likelyhood Estimation.(MLE) Maximum A Posteriori Estimation.(MAP) Numerical Problems MLE and MAP Supervised Learning Feature Representation and Problem Formulation Role of loss functions and optimization  Regression Introduction to Regression   Linear regression with one variable  (  lab experiment ) Example Problems- simple linear regression - University Questions Evaluation Metrics for Linear Regression Linear Regression...
Read more

Introduction to Machine Learning (ML)

  📘 Introduction to Machine Learning (ML) 🔹 What is Machine Learning? Machine Learning is a subfield of Artificial Intelligence that focuses on developing algorithms that allow computers to learn patterns from data and make decisions or predictions without explicit programming . It focuses on developing algorithms and models that allow machines to: Identify patterns in data Make decisions with minimal human intervention Predict future outcomes based on past experiences By continuously learning from new data, machine learning systems become more accurate and efficient over time. 📌 Formal Definition  A computer program is said to learn from experience E with respect to some task T and performance measure P if its performance at task T , as measured by P , improves with experience E . 🔹 Why Machine Learning? Traditional programming: Input + Program → Output Machine Learning paradigm: Input + Output → Model (learned program) 👉 Instead of writing...
Read more

Unsupervised Learning

Image
  📘 Unsupervised Learning 🔹 Definition Unsupervised Learning is a type of Machine Learning where the model is trained on unlabeled data , i.e., data without predefined outputs. 👉 Goal: Discover hidden patterns, structures, or relationships in data. This type of learning is particularly useful for discovering clusters in data, detecting anomalies, and finding associations. Techniques like clustering, association rule mining, and dimensionality reduction allow for deeper insights into complex datasets. It's widely used in customer segmentation, fraud detection, and market basket analysis. 🔹 Key Idea Given only input data: X = { x 1 , x 2 , x 3 , . . . , x n } X = \{x_1, x_2, x_3, ..., x_n\} 👉 The model tries to: Group similar data Identify patterns Reduce dimensionality 🔹 Characteristics No labeled outputs No “teacher” Exploratory in nature Used for pattern discovery Pic Courtesy: database town 🔹Types of Unsupervised Learning 🟦 A. Clustering ...
Read more