Implementation of XOR Problem using Neural Network in Keras

 

Experiment

Implementation of XOR Problem using Neural Network in Keras

---

🎯 Objective

• To implement a neural network using Keras
• To solve the XOR classification problem
• To visualize the decision boundary
• To understand forward propagation and learning behavior

---

🧠 Theory

---

🔹 1. XOR Problem

XOR (Exclusive OR) produces output 1 when inputs are different:

x1	x2	y
0	0	0
0	1	1
1	0	1
1	1	0

👉 XOR is not linearly separable, meaning it cannot be solved using a single straight line.

---

🔹 2. Neural Network Requirement

To solve XOR:

• A hidden layer is required
• The network learns non-linear decision boundaries

---

🔹 3. Model Architecture

• Input Layer → 2 neurons
• Hidden Layer → 4 neurons
• Output Layer → 1 neuron

---

🔹 4. Activation Functions

• Hidden layer → tanh (captures non-linearity well)
• Output layer → sigmoid (binary output)

---

🔹 5. Loss Function

• Binary Crossentropy:

$$L = -[y \log(\hat{y}) + (1-y)\log(1-\hat{y})]$$

---

🔹 6. Optimizer

• Adam optimizer:
  • Adaptive learning rate
  • Efficient training

---

🔹 7. Keras Input Layer 

Modern Keras requires an explicit input definition:

Input(shape=(2,))

👉 Avoids warnings and improves model clarity.

---

💻 Program ( Keras Implementation)

import numpy as np
import matplotlib.pyplot as plt
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Input

# XOR dataset
X = np.array([[0,0],
              [0,1],
              [1,0],
              [1,1]])

y = np.array([[0],
              [1],
              [1],
              [0]])

# Build model 
model = Sequential([
    Input(shape=(2,)),
    Dense(4, activation='tanh'),
    Dense(1, activation='sigmoid')
])

# Compile model
model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

# Train model
history = model.fit(X, y, epochs=2000, verbose=0)

# Predictions
pred = model.predict(X)
print("Predictions:\n", pred)

Predictions:
 [[0.03240691]
 [0.8563153 ]
 [0.9425467 ]
 [0.12907246]]

---

📈 Loss Curve Visualization

plt.plot(history.history['loss'])
plt.title("Loss vs Epochs")
plt.xlabel("Epochs")
plt.ylabel("Loss")
plt.show()

---

🌈 Decision Boundary Visualization

# Create grid
xx, yy = np.meshgrid(np.linspace(-0.5,1.5,200),
                     np.linspace(-0.5,1.5,200))

grid = np.c_[xx.ravel(), yy.ravel()]

# Predict probabilities
Z = model.predict(grid)
Z = Z.reshape(xx.shape)

# Plot decision boundary
plt.contourf(xx, yy, Z, levels=50, cmap=plt.cm.coolwarm)

# Plot data points
for i in range(len(X)):
    if y[i] == 0:
        plt.scatter(X[i,0], X[i,1], color='blue', edgecolors='black', s=100)
    else:
        plt.scatter(X[i,0], X[i,1], color='red', edgecolors='black', s=100)

plt.title("XOR Decision Boundary (Keras)")
plt.xlabel("x1")
plt.ylabel("x2")

plt.xlim(-0.5,1.5)
plt.ylim(-0.5,1.5)

plt.show()

---

🔍 Observations

• Loss decreases over epochs
• Model learns XOR correctly
• Decision boundary is non-linear (curved regions)

---

Result

• Neural network successfully classifies XOR inputs
• Visualization confirms non-linear separation