Determining Optimal K using Elbow Method

 

Experiment

Determining Optimal K using Elbow Method

---

🎯 Objective

To determine the optimal number of clusters (K) for a dataset using the Elbow Method.

---

📘 Theory

🔹 Why do we need Elbow Method?

In K-Means, we must predefine K, but:

• Too small K → underfitting
• Too large K → overfitting

The Elbow Method helps choose the best K.

---

🔹 What is Elbow Method?

It calculates the Within-Cluster Sum of Squares (WCSS) for different values of K.

---

🔹 WCSS Formula

$WCSS = \sum_{i=1}^{K} \sum_{x \in C_i} (x - \mu_i)^2$

---

🔹 Working Principle

1. Run K-Means for different values of K (e.g., 1 to 10)
2. Compute WCSS (inertia)
3. Plot K vs WCSS
4. Identify the “elbow point”:
   • Where WCSS starts decreasing slowly

---

🧾 Sample Dataset

Point	X	Y
P1	2	3
P2	3	4
P3	3	3
P4	8	7
P5	7	8
P6	8	8
P7	25	30

💻 Program (Python Code)

import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans

# Step 1: Dataset
X = np.array([
    [2, 3],
    [3, 4],
    [3, 3],
    [8, 7],
    [7, 8],
    [8, 8],
    [25, 30]
])

# Step 2: Compute WCSS for different K values
wcss = []

for k in range(1, 8):
    kmeans = KMeans(n_clusters=k, init='k-means++', random_state=0)
    kmeans.fit(X)
    wcss.append(kmeans.inertia_)

# Step 3: Plot the Elbow graph
plt.plot(range(1, 8), wcss, marker='o')
plt.title("Elbow Method")
plt.xlabel("Number of Clusters (K)")
plt.ylabel("WCSS")
plt.show()

# Step 4: Print values
for i, val in enumerate(wcss, start=1):
    print(f"K = {i}, WCSS = {val}")

---

📊  Output

🔹 WCSS Values (Example)

K = 1 → High value
K = 2 → Large drop
K = 3 → Moderate drop
K = 4 → Slight drop
...

K = 1, WCSS = 920.0
K = 2, WCSS = 68.33333333333333
K = 3, WCSS = 2.666666666666667
K = 4, WCSS = 1.8333333333333335
K = 5, WCSS = 1.0
K = 6, WCSS = 0.5
K = 7, WCSS = 0.0

---

📈 Graph Interpretation

• The curve drops sharply initially
• After a certain K, the decrease slows
• The “bend” or elbow point indicates optimal K

👉 For this dataset, the elbow is typically at:

✅ K = 3

---

📌 Result

The optimal number of clusters was determined using the Elbow Method, and the best value of K is found to be:

K = 3

• Elbow Method helps select appropriate K
• Uses WCSS as evaluation metric
• Easy to implement and visualize