Comparison of K-Means and Agglomerative Clustering on Mall Customers Dataset

 

Experiment

Comparison of K-Means and Agglomerative Clustering on Mall Customers Dataset

---

🎯 Objective

To apply and compare:

• K-Means (Partitional Clustering)
• Agglomerative (Hierarchical Clustering)

using evaluation metrics and visualization.

---

📘 Theory

🔹 Partitional Clustering (K-Means)

• Divides data into K clusters
• Minimizes within-cluster variance

---

🔹 Hierarchical Clustering (Agglomerative)

• Builds clusters bottom-up
• Produces a tree structure (dendrogram)

---

🔹 Evaluation Metrics

1. Inertia (WCSS)

$WCSS = \sum_{i=1}^{K} \sum_{x \in C_i} (x - \mu_i)^2$

• Used for K-Means only

---

2. Silhouette Score

$s(i) = \frac{b(i) - a(i)}{\max(a(i), b(i))}$

• Used for both methods
• Higher value → better clustering

---

🧾 Dataset: Mall Customers

Typical features:

• CustomerID
• Gender
• Age
• Annual Income (k$)
• Spending Score (1–100)

👉 We use:

• Annual Income
• Spending Score

---

💻 Program (Python Code)

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans, AgglomerativeClustering
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler
from scipy.cluster.hierarchy import dendrogram, linkage

# -------------------------------
# Step 1: Load Dataset
# -------------------------------
# Replace with your file path
data = pd.read_csv("Mall_Customers.csv")

# Select features
X = data[['Annual Income (k$)', 'Spending Score (1-100)']]

# -------------------------------
# Step 2: Preprocessing
# -------------------------------
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# -------------------------------
# Step 3: K-Means Clustering
# -------------------------------
k = 5

kmeans = KMeans(n_clusters=k, random_state=0)
k_labels = kmeans.fit_predict(X_scaled)

k_inertia = kmeans.inertia_
k_silhouette = silhouette_score(X_scaled, k_labels)

# -------------------------------
# Step 4: Agglomerative Clustering
# -------------------------------
agg = AgglomerativeClustering(n_clusters=k, linkage='ward')
a_labels = agg.fit_predict(X_scaled)

a_silhouette = silhouette_score(X_scaled, a_labels)

# -------------------------------
# Step 5: Visualization
# -------------------------------
plt.figure()
plt.scatter(X_scaled[:, 0], X_scaled[:, 1], c=k_labels)
plt.title("K-Means Clustering")
plt.xlabel("Income")
plt.ylabel("Spending Score")
plt.show()

plt.figure()
plt.scatter(X_scaled[:, 0], X_scaled[:, 1], c=a_labels)
plt.title("Agglomerative Clustering")
plt.xlabel("Income")
plt.ylabel("Spending Score")
plt.show()

# -------------------------------
# Step 6: Dendrogram
# -------------------------------
Z = linkage(X_scaled, method='ward')

plt.figure()
dendrogram(Z)
plt.title("Dendrogram")
plt.xlabel("Customers")
plt.ylabel("Distance")
plt.show()

# -------------------------------
# Step 7: Results
# -------------------------------
print("K-Means Inertia:", k_inertia)
print("K-Means Silhouette:", k_silhouette)
print("Agglomerative Silhouette:", a_silhouette)

---

📈 Output

K-Means Inertia: 65.56840815571681 K-Means Silhouette: 0.5546571631111091 Agglomerative Silhouette: 0.5538089226688662

📊  Observations

🔹 K-Means

• Produces compact, spherical clusters
• Faster and scalable

---

🔹 Agglomerative

• Captures hierarchical relationships
• More flexible cluster shapes

---

Method	Inertia	Silhouette
K-Means	Low	~0.55
Agglomerative	—	~0.55

---

📌 Result

Both clustering algorithms were applied successfully.
K-Means showed slightly better compactness, while Agglomerative provided hierarchical insights.

---

⚖️ Comparison

Feature	K-Means	Agglomerative
Type	Partitional	Hierarchical
Speed	Fast	Slow
Scalability	High	Low
Shape handling	Spherical	Flexible
Output	Flat clusters	Tree (dendrogram)

• K-Means
  • Efficient for large datasets
  • Requires predefined K
• Agglomerative
  • More interpretable (dendrogram)
  • Computationally expensive

👉 Choice depends on:

• Dataset size
• Cluster shape
• Need for hierarchy