Implementation of Ridge and Lasso Regression on Diabetes Dataset with Hyperparameter Tuning

Experiment 

Implementation of Ridge and Lasso Regression on Diabetes Dataset with Hyperparameter Tuning

---

🎯 Aim

To implement and compare Linear, Ridge, and Lasso Regression using evaluation metrics and visualize their performance.

---

Objectives

• Load and preprocess dataset
• Implement Linear, Ridge, and Lasso regression
• Tune hyperparameters using cross-validation
• Compare using MSE and R²
• Visualize predictions and coefficient shrinkage

---

📖 Additional Visualization Insight

We will plot:

1. Actual vs Predicted values
2. Coefficient comparison (shrinkage effect)

---

💻 Program 

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression, RidgeCV, LassoCV
from sklearn.metrics import mean_squared_error, r2_score
# -----------------------------
# Load Dataset
# -----------------------------
data = load_diabetes()
print(data.DESCR)
df = pd.DataFrame(data.data, columns=data.feature_names)
df['target'] = data.target
# -----------------------------
# Feature and Target
# -----------------------------
X = df.drop('target', axis=1)
y = df['target']
# -----------------------------
# Train-Test Split
# -----------------------------
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
# -----------------------------
# Linear Regression
# -----------------------------
lin_model = LinearRegression()
lin_model.fit(X_train, y_train)
y_pred_lin = lin_model.predict(X_test)
# -----------------------------
# Ridge Regression
# -----------------------------
ridge_alphas = np.logspace(-3, 3, 50)
ridge_model = RidgeCV(alphas=ridge_alphas, cv=5)
ridge_model.fit(X_train, y_train)
y_pred_ridge = ridge_model.predict(X_test)
# -----------------------------
# Lasso Regression
# -----------------------------
lasso_model = LassoCV(cv=5, max_iter=10000)
lasso_model.fit(X_train, y_train)
y_pred_lasso = lasso_model.predict(X_test)
# -----------------------------
# Evaluation
# -----------------------------
models = {
    "Linear": y_pred_lin,
    "Ridge": y_pred_ridge,
    "Lasso": y_pred_lasso
}
print("\nModel Performance:")
print("----------------------------------")
print("Model\t\tMSE\t\tR²")
print("----------------------------------")
for name, y_pred in models.items():
    mse = mean_squared_error(y_test, y_pred)
    r2 = r2_score(y_test, y_pred)
    print(f"{name}\t\t{mse:.4f}\t{r2:.4f}")
# -----------------------------
# Graph 1: Actual vs Predicted
# -----------------------------
plt.figure()
plt.scatter(y_test, y_pred_lin, label="Linear")
plt.scatter(y_test, y_pred_ridge, label="Ridge")
plt.scatter(y_test, y_pred_lasso, label="Lasso")
plt.xlabel("Actual Values")
plt.ylabel("Predicted Values")
plt.title("Actual vs Predicted Comparison")
plt.legend()
plt.show()
# -----------------------------
# Graph 2: Coefficient Comparison
# -----------------------------
coef_df = pd.DataFrame({
    'Feature': X.columns,
    'Linear': lin_model.coef_,
    'Ridge': ridge_model.coef_,
    'Lasso': lasso_model.coef_
})

print("\nCoefficients:")
print(coef_df)
coef_df.set_index('Feature').plot(kind='bar')
plt.title("Coefficient Comparison (Shrinkage Effect)")
plt.ylabel("Coefficient Value")
plt.xticks(rotation=45)
plt.show()
# -----------------------------
# Best Alpha Values
# -----------------------------
print("\nBest Alpha Values:")
print("Ridge Alpha:", ridge_model.alpha_)
print("Lasso Alpha:", lasso_model.alpha_)

📊 Sample Output (Typical)

Model Performance:
----------------------------------
Model		MSE		R²
----------------------------------
Linear		2900.1936	0.4526
Ridge		2857.6963	0.4606
Lasso		2800.2627	0.4715

Coefficients: Feature Linear Ridge Lasso 0 age 37.904021 42.758013 0.000000 1 sex -241.964362 -208.277776 -168.165267 2 bmi 542.428759 509.219395 554.134694 3 bp 347.703844 319.219682 311.629705 4 s1 -931.488846 -115.066100 -101.862897 5 s2 518.062277 -84.195275 -0.000000 6 s3 163.419983 -188.950272 -235.137459 7 s4 275.317902 153.017313 0.000000 8 s5 736.198859 397.153387 460.585713 9 s6 48.670657 78.115291 36.922057

Best Alpha Values: Ridge Alpha: 0.09102981779915217 Lasso Alpha: 0.07813983904476526

---

📊 Graph Explanation

📈 Graph 1: Actual vs Predicted

• Points closer to diagonal → better predictions
• Ridge & Lasso usually cluster better than Linear

---

📉 Graph 2: Coefficient Comparison

• Linear → large coefficients
• Ridge → shrunk coefficients
• Lasso → some coefficients become zero

---

📈 Interpretation

• Linear Regression
  • Baseline model
  • No regularization
• Ridge Regression
  • Slightly better performance
  • Reduces overfitting
• Lasso Regression
  • Performs feature selection
  • Some coefficients become zero

---

📊 Observations

Model	Behavior
Linear	No regularization
Ridge	Stable, better generalization
Lasso	Sparse model

---

Result

Ridge and Lasso regression models were successfully implemented and tuned using cross-validation, showing improved or comparable performance to standard linear regression.

• Regularization improves model generalization
• Ridge works well when all features are important
• Lasso helps in feature selection
• Cross-validation is essential for choosing alpha
• Visualization helps understand model behavior clearly