Lab Assignment-9

 

Assignment-9- week 3 September

Learning Objective: Learn SVM

1.Implementation of Linear SVM from Scratch

Problem Statement

A college wants to classify students into two categories—Pass and Fail—based on their performance in two internal assessment tests.

Each student is represented using two features:

• Internal Test 1 Score
• Internal Test 2 Score

The target class is:

• −1 → Fail
• +1 → Pass

The following training dataset is provided.

Student	Test 1	Test 2	Class
S1	2	3	−1
S2	3	2	−1
S3	3	4	−1
S4	4	3	−1
S5	6	7	+1
S6	7	6	+1
S7	7	8	+1
S8	8	7	+1

Task

Implement a Linear Support Vector Machine (SVM) classifier from scratch using Python and NumPy, without using Scikit-learn's SVC or any other built-in SVM implementation.

Your program should:

1. Represent the input data using NumPy arrays.
2. Visualize the two classes using a scatter plot.
3. Initialize the weight vector w and bias b.
4. Implement the linear decision function:

$$f(x)=w^Tx+b$$

5. Classify samples using the sign of the decision function.
6. Implement the SVM training process using an appropriate optimization approach such as gradient descent/sub-gradient descent.
7. Train the model to find the optimal values of w and b that maximize the margin between the two classes.
8. Identify the support vectors.
9. Plot:
   • The two classes
   • The separating hyperplane
   • The margin boundaries
   • The support vectors
10. Use the trained SVM model to predict the classes of the following new students:

Student	Test 1	Test 2
A	4	5
B	7	7
C	5	5
D	2	4

11. Display the predicted class for each student.

---

Additional Experiment: Effect of the Regularization Parameter

Modify the program to introduce the regularization parameter C and experiment with:

• C = 0.1
• C = 1
• C = 10
• C = 100

For each value of C:

1. Train the SVM model.
2. Plot the decision boundary and margins.
3. Count the number of misclassified samples.
4. Compare the margin obtained.
5. Discuss the effect of C on margin size and classification errors.

2.Implement a Linear Support Vector Machine (SVM) to classify the Iris dataset. Visualize the decision boundary and discuss how the margin is determined.

Tasks:

● Load and preprocess the Iris dataset.

● Implement a Linear SVM for binary classification (e.g., classify Setosa vs. Non-

Setosa).

● Visualize the decision boundary and margin.

● Discuss the concept of the margin and how it influences classification.

3.Implement and compare the performance of SVM classifiers with linear, polynomial, and RBF kernels on the Fashion MNIST dataset. Analyze the advantages and disadvantages of each kernel type.

Tasks:

● Load and preprocess the Fashion MNIST dataset.

● Implement SVM with linear, polynomial, and RBF kernels.

● Compare the classification performance for each kernel.

● Discuss the strengths and weaknesses of each kernel type.