Classification of Non-Linear Data using SVM with RBF Kernel

 

Experiment 

Classification of Non-Linear Data using SVM with RBF Kernel

---

🎯 Objective

To implement Support Vector Machine (SVM) with RBF kernel to classify non-linearly separable data and visualize the decision boundary.

---

📚 Theory

Some datasets cannot be separated using a straight line. Such data is called non-linearly separable.

🔹 Solution: Kernel Trick

SVM uses a kernel function to map data into a higher-dimensional space where it becomes linearly separable.

🔹 RBF Kernel (Radial Basis Function)

The most commonly used kernel:

$K(x,x^{\mathrm{\prime }})=\exp (-\gamma \mathrm{\parallel }x-x^{\mathrm{\prime }}{\mathrm{\parallel }}^2)$

Where:

• $\gamma$ controls how far influence of a point reaches
• Small γ → smooth boundary
• Large γ → complex boundary

---

📊 Dataset

We generate a non-linear circular dataset using make_circles:

• Inner circle → Class 0
• Outer circle → Class 1

---

🔬 Algorithm / Procedure

1. Import required libraries
2. Generate non-linear dataset
3. Train SVM with RBF kernel
4. Plot dataset
5. Plot decision boundary
6. Analyze results

---

💻 Program

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm
from sklearn.datasets import make_circles

# Generate non-linear dataset
X, y = make_circles(n_samples=100, noise=0.1, factor=0.5, random_state=42)

# Create SVM model with RBF kernel
model = svm.SVC(kernel='rbf', gamma=1,C=2.0)

# Train model
model.fit(X, y)

# Plot data points
plt.scatter(X[:, 0], X[:, 1], c=y)

# Plot decision boundary
ax = plt.gca()
xlim = ax.get_xlim()
ylim = ax.get_ylim()

# Create grid
xx = np.linspace(xlim[0], xlim[1], 100)
yy = np.linspace(ylim[0], ylim[1], 100)
YY, XX = np.meshgrid(yy, xx)
xy = np.vstack([XX.ravel(), YY.ravel()]).T

Z = model.decision_function(xy).reshape(XX.shape)

# Plot decision boundary
ax.contour(XX, YY, Z, levels=[0])

# Plot margins
ax.contour(XX, YY, Z, levels=[-1, 1], linestyles=['--', '--'])

# Highlight support vectors
ax.scatter(model.support_vectors_[:, 0],
           model.support_vectors_[:, 1],
           s=100, facecolors='none', edgecolors='k')

plt.title("SVM with RBF Kernel (Non-linear Data)")
plt.xlabel("x1")
plt.ylabel("x2")
plt.show()

---

📈 Output

• Circular dataset plotted
• Non-linear decision boundary (curved line)
• Margins shown
• Support vectors highlighted

---

✅ Result

The SVM with RBF kernel successfully classified non-linear data by creating a non-linear decision boundary.

A non-linear decision boundary was observed, demonstrating the effectiveness of the kernel trick.

The support vectors were identified and visualized; these points lie closest to the decision boundary and play a crucial role in defining it.

The margin boundaries were also visualized, showing the region of separation between classes.