Logistic Regression on Pima Indians Diabetes Dataset

 

Experiment

Logistic Regression on Pima Indians Diabetes Dataset

🎯 Aim

To implement Logistic Regression for disease prediction and compare model performance with and without feature scaling.

---

📘 Theory 

Logistic Regression predicts probability using:

$$h(x) = \frac{1}{1 + e^{-z}}, \quad z = \theta^T x$$

• Output ≥ 0.5 → Diabetic(diabetes = 1)
• Output < 0.5 → No disease (0)

---

📊 Dataset Description

The Pima Indians Diabetes Dataset contains medical attributes:

Feature	Description
Pregnancies	Number of pregnancies
Glucose	Blood glucose level
BloodPressure	Blood pressure
SkinThickness	Skin fold thickness
Insulin	Insulin level
BMI	Body mass index
DiabetesPedigreeFunction	Genetic influence
Age	Age
Outcome	0 (No diabetes), 1 (Diabetes)

---

⚙️ Procedure

1. Load dataset
2. Split into training and testing sets
3. Train model without scaling
4. Evaluate performance
5. Apply feature scaling
6. Train again
7. Compare results

💻 Program (Scikit-learn Implementation)

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from sklearn.preprocessing import StandardScaler

# Load dataset
data = pd.read_csv("diabetes.csv")

# Features and target
X = data.iloc[:, :-1]
y = data.iloc[:, -1]

# Train-test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# -----------------------------------
# WITHOUT FEATURE SCALING
# -----------------------------------
model_no_scale = LogisticRegression(max_iter=1000)
model_no_scale.fit(X_train, y_train)

y_pred_no_scale = model_no_scale.predict(X_test)

metrics_no_scale = {
    "Accuracy": accuracy_score(y_test, y_pred_no_scale),
    "Precision": precision_score(y_test, y_pred_no_scale),
    "Recall": recall_score(y_test, y_pred_no_scale),
    "F1 Score": f1_score(y_test, y_pred_no_scale)
}

print("Without Scaling:")
print(metrics_no_scale)

# -----------------------------------
# WITH FEATURE SCALING
# -----------------------------------
scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model_scaled = LogisticRegression(max_iter=1000)
model_scaled.fit(X_train_scaled, y_train)

y_pred_scaled = model_scaled.predict(X_test_scaled)

metrics_scaled = {
    "Accuracy": accuracy_score(y_test, y_pred_scaled),
    "Precision": precision_score(y_test, y_pred_scaled),
    "Recall": recall_score(y_test, y_pred_scaled),
    "F1 Score": f1_score(y_test, y_pred_scaled)
}

print("\nWith Scaling:")
print(metrics_scaled)

📈 Expected Output 

Without Scaling: {'Accuracy': 0.609375, 'Precision': 0.46208530805687204, 'Recall': 0.7276119402985075, 'F1 Score': 0.5652173913043478}
With Scaling: {'Accuracy': 0.78125, 'Precision': 0.7358490566037735, 'Recall': 0.582089552238806, 'F1 Score': 0.65}

Result

❌ Without Scaling

• Features with large values dominate (e.g., Glucose)
• Slower convergence of solver
• Suboptimal decision boundary

✅ With Scaling

• Balanced contribution of features
• Faster convergence
• Improved generalization

---

• Scikit-learn simplifies logistic regression implementation
• Feature scaling significantly improves:
  • Accuracy
  • Precision & Recall
  • Model stability