Implementation of XOR Problem using Neural Network in scikit-learn

 

Experiment

Implementation of XOR Problem using Neural Network in scikit-learn

---

🎯 Objective

• To implement a neural network using scikit-learn
• To solve the XOR classification problem
• To visualize the decision boundary

---

Theory

---

🔹 1. XOR Problem

XOR (Exclusive OR) outputs 1 when inputs are different:

x1	x2	y
0	0	0
0	1	1
1	0	1
1	1	0

---

🔹 2. Key Concept

• XOR is not linearly separable
• A single-layer model cannot solve it
• Requires a multi-layer neural network

---

🔹 3. Neural Network (MLPClassifier)

We use:

• Input layer: 2 neurons
• Hidden layer: 2 neurons
• Output layer: 1 neuron

---

🔹 4. Activation Function

• Sigmoid (logistic) function:

$$\sigma(x) = \frac{1}{1 + e^{-x}}$$

🔹 5. Optimizer

• lbfgs is used:
  • Fast convergence
  • Suitable for small datasets

---

💻 Program

import numpy as np
import matplotlib.pyplot as plt
from sklearn.neural_network import MLPClassifier

# XOR dataset
X = np.array([[0,0],
              [0,1],
              [1,0],
              [1,1]])

y = np.array([0, 1, 1, 0])

# Create model
model = MLPClassifier(hidden_layer_sizes=(2,),
                      activation='logistic',
                      solver='lbfgs',
                      max_iter=1000,
                      random_state=0)

# Train model
model.fit(X, y)

# Predictions
predictions = model.predict(X)
print("Predictions:", predictions)

Output:

Predictions: [0 1 1 0]

---

🌈 Decision Boundary Visualization

# Create grid
xx, yy = np.meshgrid(np.linspace(-0.5,1.5,200),
                     np.linspace(-0.5,1.5,200))

grid = np.c_[xx.ravel(), yy.ravel()]
Z = model.predict(grid)
Z = Z.reshape(xx.shape)

# Plot decision boundary
plt.contourf(xx, yy, Z, alpha=0.4, cmap=plt.cm.coolwarm)

# Plot data points
for i in range(len(X)):
    if y[i] == 0:
        plt.scatter(X[i,0], X[i,1], color='blue', edgecolors='black', s=100)
    else:
        plt.scatter(X[i,0], X[i,1], color='red', edgecolors='black', s=100)

plt.title("XOR Decision Boundary (scikit-learn)")
plt.xlabel("x1")
plt.ylabel("x2")

plt.xlim(-0.5,1.5)
plt.ylim(-0.5,1.5)

plt.show()

---

🔍 Observations

• Model correctly classifies XOR outputs
• Decision boundary is non-linear
• Hidden layer enables separation

---

Result

• Neural network successfully solves XOR problem
• Visualization shows the  decision boundary