Implementation of linear SVM using Scikit-learn with Visualization

 

Experiment 

Implementation of linear SVM using Scikit-learn with Visualization

---

🎯 Objective

To implement Support Vector Machine (SVM) using Scikit-learn and visualize the decision boundary and support vectors.

---

📚 Theory

Support Vector Machine (SVM) is a supervised learning algorithm used for classification. It finds the optimal hyperplane that separates classes with maximum margin.

Key Points:

• Works well for linearly separable data
• Uses support vectors to define boundary
• Can use different kernels:
  • Linear
  • Polynomial
  • RBF

---

📊 Dataset

We use a simple 2D dataset:

x₁	x₂	Class
2	2	0
3	3	0
4	4	0
6	6	1
7	7	1
8	8	1

---

🔬 Algorithm / Procedure

1. Import libraries
2. Create dataset
3. Train SVM model using linear kernel
4. Plot data points
5. Plot decision boundary
6. Highlight support vectors

---

💻 Program

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm

# Dataset
X = np.array([[2,2], [3,3], [4,4], [6,6], [7,7], [8,8]])
y = np.array([0, 0, 0, 1, 1, 1])

# Create SVM model
model = svm.SVC(kernel='linear')

# Train model
model.fit(X, y)

# Plot data points
plt.scatter(X[:,0], X[:,1], c=y, cmap='coolwarm')

# Plot decision boundary
ax = plt.gca()
xlim = ax.get_xlim()
ylim = ax.get_ylim()

# create grid
xx = np.linspace(xlim[0], xlim[1], 30)
yy = np.linspace(ylim[0], ylim[1], 30)
YY, XX = np.meshgrid(yy, xx)
xy = np.vstack([XX.ravel(), YY.ravel()]).T

Z = model.decision_function(xy).reshape(XX.shape)

# plot decision boundary and margins
ax.contour(XX, YY, Z, levels=[0])
ax.contour(XX, YY, Z, levels=[-1, 1], linestyles=['--', '--'])

# highlight support vectors
ax.scatter(model.support_vectors_[:,0],
           model.support_vectors_[:,1],
           s=100, facecolors='none', edgecolors='k')

plt.xlabel("x1")
plt.ylabel("x2")
plt.title("SVM Decision Boundary with Support Vectors")
plt.show()

---

📈 Output

• Scatter plot of data points
• Decision boundary (solid line)
• Margin lines (dashed)
• Support vectors (circled points)

---

✅ Result

The SVM classifier was successfully implemented using Scikit-learn, and the decision boundary along with support vectors was visualized