Logistic Regression with assessments Confusion Matrix ,ROC and AUC

 

Experiment: 

Logistic Regression with assessments Confusion Matrix ,ROC amd AUC

---

🎯 Aim

To implement Logistic Regression using scikit-learn on the Pima Indians Diabetes dataset and evaluate model performance with and without feature scaling using:

• Accuracy
• Precision
• Recall
• F1-score
• Confusion Matrix
• ROC Curve and AUC

---

📘 Theory

Logistic Regression is a supervised classification algorithm used for binary classification problems. It predicts the probability of a class using the sigmoid function:

$$P(y=1|x) = \frac{1}{1 + e^{-\theta^T x}}$$

• Output ranges from 0 to 1
• A threshold (0.5) is used to classify:
  • ≥ 0.5 → Class 1 (Disease present)
  • < 0.5 → Class 0 (No disease)

---

🔹 Feature Scaling

Feature scaling standardizes input features so that all variables contribute equally:

$$x' = \frac{x - \mu}{\sigma}$$

• Helps faster convergence
• Improves model performance

---

🔹 Confusion Matrix

	Predicted 0	Predicted 1
Actual 0	TN	FP
Actual 1	FN	TP

---

ROC Curve (Receiver Operating Characteristic)

The ROC Curve is a graphical representation used to evaluate the performance of a binary classification model at different threshold values.

Instead of fixing the threshold at 0.5, the ROC curve shows how the model behaves for all possible thresholds (0 to 1).

---

🔸 Key Terms

• True Positive Rate (TPR) / Recall / Sensitivity

$$TPR = \frac{TP}{TP + FN}$$

• False Positive Rate (FPR)

$$FPR = \frac{FP}{FP + TN}$$

---

🔸 ROC Curve Interpretation

• X-axis → False Positive Rate (FPR)
• Y-axis → True Positive Rate (TPR)

Each point on the curve represents a different threshold value.

---

🔸 Important Observations

• A curve closer to the top-left corner indicates better performance
• A random model gives a diagonal line (no discrimination ability)
• A perfect model reaches the point (0, 1)

---

🔹 AUC (Area Under the Curve)

The AUC is the area under the ROC curve and provides a single number to summarize model performance.

---

🔸 AUC Interpretation

AUC Value	Meaning
1.0	Perfect classifier
0.9 – 0.99	Excellent
0.8 – 0.89	Good
0.7 – 0.79	Fair
0.5	Random guessing
< 0.5	Poor model

---

🔸 Intuitive Meaning of AUC

AUC represents the probability that the model ranks a random positive example higher than a random negative example.

---

🔹 Why ROC–AUC is Important

• Works well even when classes are imbalanced
• Independent of classification threshold
• Gives a comprehensive evaluation of model performance

---

🔹 In This Experiment

• ROC curves are plotted for:
  • Model without scaling
  • Model with scaling
• AUC values are compared to determine which model performs better

---

 Simple Insight for Students

ROC shows how good the model is at separating classes,
AUC tells how good it is overall in one number.

📊 Dataset

Pima Indians Diabetes Dataset ( used in the previous experiment)

• Medical diagnostic dataset
• Binary target:
  • 0 → No Diabetes
  • 1 → Diabetes

---

⚙️ Procedure

1. Load the dataset
2. Separate features and target
3. Split into training and testing sets
4. Train Logistic Regression model without scaling
5. Evaluate using metrics, confusion matrix, and ROC curve
6. Apply feature scaling
7. Train model again
8. Evaluate and compare results

---

💻 Program

import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (accuracy_score, precision_score, recall_score, 
                             f1_score, confusion_matrix, ConfusionMatrixDisplay,
                             roc_curve, roc_auc_score)
from sklearn.preprocessing import StandardScaler

# Load dataset
data = pd.read_csv("diabetes.csv")

# Features and target
X = data.iloc[:, :-1]
y = data.iloc[:, -1]

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# -----------------------------------
# WITHOUT FEATURE SCALING
# -----------------------------------
model_no_scale = LogisticRegression(max_iter=1000)
model_no_scale.fit(X_train, y_train)

y_pred_no_scale = model_no_scale.predict(X_test)
y_prob_no_scale = model_no_scale.predict_proba(X_test)[:, 1]

print("Without Scaling:")
print("Accuracy:", accuracy_score(y_test, y_pred_no_scale))
print("Precision:", precision_score(y_test, y_pred_no_scale))
print("Recall:", recall_score(y_test, y_pred_no_scale))
print("F1 Score:", f1_score(y_test, y_pred_no_scale))

# Confusion Matrix
ConfusionMatrixDisplay.from_predictions(y_test, y_pred_no_scale)
plt.title("Confusion Matrix (Without Scaling)")
plt.show()

# ROC Curve
fpr_ns, tpr_ns, _ = roc_curve(y_test, y_prob_no_scale)
auc_ns = roc_auc_score(y_test, y_prob_no_scale)

plt.figure()
plt.plot(fpr_ns, tpr_ns, label=f"AUC = {auc_ns:.2f}")
plt.plot([0,1], [0,1], linestyle='--')
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("ROC Curve (Without Scaling)")
plt.legend()
plt.show()

# -----------------------------------
# WITH FEATURE SCALING
# -----------------------------------
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model_scaled = LogisticRegression(max_iter=1000)
model_scaled.fit(X_train_scaled, y_train)

y_pred_scaled = model_scaled.predict(X_test_scaled)
y_prob_scaled = model_scaled.predict_proba(X_test_scaled)[:, 1]

print("\nWith Scaling:")
print("Accuracy:", accuracy_score(y_test, y_pred_scaled))
print("Precision:", precision_score(y_test, y_pred_scaled))
print("Recall:", recall_score(y_test, y_pred_scaled))
print("F1 Score:", f1_score(y_test, y_pred_scaled))

# Confusion Matrix
ConfusionMatrixDisplay.from_predictions(y_test, y_pred_scaled)
plt.title("Confusion Matrix (With Scaling)")
plt.show()

# ROC Curve
fpr_s, tpr_s, _ = roc_curve(y_test, y_prob_scaled)
auc_s = roc_auc_score(y_test, y_prob_scaled)

plt.figure()
plt.plot(fpr_s, tpr_s, label=f"AUC = {auc_s:.2f}")
plt.plot([0,1], [0,1], linestyle='--')
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("ROC Curve (With Scaling)")
plt.legend()
plt.show()

---

Output

📈 Result

• Logistic Regression model was successfully implemented
• Performance metrics were computed
• Confusion matrices and ROC curves were plotted
• Model with feature scaling showed improved performance

---

🔍 Inference

• Feature scaling improves:
  • Model accuracy
  • Convergence speed
  • ROC-AUC score
• It ensures fair contribution of all features