K-Means Clustering on Digits Dataset with Performance Evaluation

 

Experiment

K-Means Clustering on Digits Dataset with Performance Evaluation

---

🎯 Objective

To apply K-Means clustering on the Digits dataset and evaluate clustering performance using:

• Inertia (WCSS)
• Silhouette Score

---

📘 Theory

🔹 Digits Dataset

• Contains 1797 samples of handwritten digits (0–9)
• Each image is 8×8 pixels (64 features)
• Used for classification and clustering tasks

---

🔹 K-Means Clustering

Partitions data into K clusters by minimizing within-cluster variance.

---

🔹 Evaluation Metrics

1. Inertia (WCSS)

$WCSS = \sum_{i=1}^{K} \sum_{x \in C_i} (x - \mu_i)^2$

• Lower value → better clustering
• Always decreases as K increases

---

2. Silhouette Score

$s(i) = \frac{b(i) - a(i)}{\max(a(i), b(i))}$

• Range: -1 to 1
• Higher value → better clustering

---

💻 Program (Python Code)

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

# Step 1: Load dataset
digits = load_digits()
X = digits.data

# Step 2: Preprocessing (Scaling)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Step 3: Try different K values
K_range = range(2, 15)
inertia_values = []
silhouette_scores = []

for k in K_range:
    kmeans = KMeans(n_clusters=k, init='k-means++', random_state=0)
    labels = kmeans.fit_predict(X_scaled)
    
    # Inertia
    inertia_values.append(kmeans.inertia_)
    
    # Silhouette Score
    score = silhouette_score(X_scaled, labels)
    silhouette_scores.append(score)

# Step 4: Plot Inertia (Elbow Method)
plt.figure()
plt.plot(K_range, inertia_values, marker='o')
plt.title("Elbow Method (Inertia vs K)")
plt.xlabel("Number of Clusters (K)")
plt.ylabel("Inertia")
plt.show()

# Step 5: Plot Silhouette Scores
plt.figure()
plt.plot(K_range, silhouette_scores, marker='o')
plt.title("Silhouette Score vs K")
plt.xlabel("Number of Clusters (K)")
plt.ylabel("Silhouette Score")
plt.show()

# Step 6: Print values
for k, inertia, score in zip(K_range, inertia_values, silhouette_scores):
    print(f"K={k}, Inertia={inertia:.2f}, Silhouette={score:.4f}")

Output
K=2, Inertia=100756.83, Silhouette=0.0894
K=3, Inertia=93840.29, Silhouette=0.1059
K=4, Inertia=89113.87, Silhouette=0.0954
K=5, Inertia=85099.65, Silhouette=0.1032
K=6, Inertia=81695.77, Silhouette=0.1055
K=7, Inertia=78934.32, Silhouette=0.1044
K=8, Inertia=75211.96, Silhouette=0.1339
K=9, Inertia=72028.05, Silhouette=0.1372
K=10, Inertia=69749.08, Silhouette=0.1455
K=11, Inertia=67274.44, Silhouette=0.1533
K=12, Inertia=66480.33, Silhouette=0.1448
K=13, Inertia=63638.62, Silhouette=0.1534
K=14, Inertia=63907.30, Silhouette=0.1510

---

📊 Expected Observations

🔹 Inertia Plot

• Decreases continuously as K increases
• Elbow typically around:

👉 K ≈ 8 to 10

---

🔹 Silhouette Plot

• Peaks at a certain K
• Often lower for very high K

👉 May not be exactly 10 (true classes)

---

📈 Analysis

🔹 Effect of K on Inertia

• K ↑ → Inertia ↓
• Reason: more clusters → tighter grouping

---

🔹 Effect of K on Silhouette Score

• Initially increases
• Then decreases after optimal K

---

🔹 Important Insight

👉 Even though dataset has 10 digits, optimal K may differ because:

• K-Means assumes spherical clusters
• Digits may overlap in feature space

---

📌 Result

• K-Means clustering was applied to the Digits dataset
• Optimal K identified using:
  • Elbow Method (Inertia)
  • Silhouette Score

👉 Observed optimal K ≈ 8–10

• Inertia alone is not sufficient
• Silhouette provides better cluster validation
• Real-world datasets may not give perfect K