Polynomial Regression with Model Evaluation (MSE & R²)

 

Experiment 

Polynomial Regression with Model Evaluation (MSE & R²)

---

🎯 Aim

To implement Polynomial Regression with varying degrees and evaluate model performance using Mean Squared Error (MSE) and R-squared (R²).

---

Objectives

• Generate sample dataset

• Fit polynomial models of different degrees

• Compute MSE and R² for each model

• Compare model performance

• Visualize regression curves

---

🛠️ Tools Required

• Python

• NumPy

• Matplotlib

• Scikit-learn

---

📖 Theory

🔹 Polynomial Regression

$$y = \theta_0 + \theta_1 x + \theta_2 x^2 + \dots + \theta_n$$

---

🔹 Model Evaluation

Mean Squared Error (MSE)

• Measures average squared error

• Lower value → better fit

R-squared (R²)

• Measures variance explained

• Value closer to 1 → better model

---

📋 Procedure

1. Generate dataset

2. Fit models for different degrees

3. Compute predictions

4. Calculate MSE and R²

5. Plot curves

6. Compare results

---

💻 Program

import numpy as np
import matplotlib.pyplot as plt

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

# -----------------------------
# Generate toy dataset
# -----------------------------
np.random.seed(42)

X = np.linspace(-3, 3, 100).reshape(-1, 1)
y = 0.5 * X**2 + X + 2 + np.random.randn(100, 1)

# -----------------------------
# Degrees to test
# -----------------------------
degrees = [1, 2, 3, 5, 10,15]

plt.figure(figsize=(12, 8))

print("Degree\tMSE\t\tR²")
print("--------------------------------")

for i, degree in enumerate(degrees, 1):
    
    # Polynomial transformation
    poly = PolynomialFeatures(degree=degree)
    X_poly = poly.fit_transform(X)
    
    # Train model
    model = LinearRegression()
    model.fit(X_poly, y)
    
    # Predictions
    y_pred = model.predict(X_poly)
    
    # Evaluation
    mse = mean_squared_error(y, y_pred)
    r2 = r2_score(y, y_pred)
    
    print(f"{degree}\t{mse:.4f}\t{r2:.4f}")
    
    # Plot
    plt.subplot(2, 3, i)
    plt.scatter(X, y, s=10)
    plt.plot(X, y_pred)
    
    plt.title(f"Degree = {degree}")
    plt.xlabel("X")
    plt.ylabel("y")

plt.tight_layout()
plt.show()

---

📊Output 

Degree	MSE	     R²
--------------------------------
1	2.8403	    0.5300
2	0.8118	    0.8657
3	0.7724	    0.8722
5	0.7629	    0.8738
10	0.7313	    0.8790
15	0.7034	    0.8836

---

📈 Interpretation

• Degree 1 → Underfitting

• Degree 2–3 → Good fit

• Degree 10 → Overfitting (too complex)

---

Result

Polynomial regression models of varying degrees were implemented and evaluated using MSE and R².

• Increasing degree reduces training error

• Very high degree leads to overfitting

• Best model balances bias and variance