Implement linear regression using scikit-learn on California Housing dataset (multiple variables)

 

Experiment 

Multiple Linear Regression using Scikit-learn on California Housing Dataset (All Features)

---

🎯 Aim

To implement Multiple Linear Regression using Scikit-learn on the California Housing dataset using all input features to predict house prices.

---

🎯 Objectives

• Load dataset from CSV

• Perform preprocessing (handling missing values + encoding)

• Use all features for prediction

• Train model using Scikit-learn

• Evaluate using MSE and R²

• Analyze feature importance

---

🛠️ Tools Required

• Python

• NumPy

• Pandas

• Matplotlib

• Scikit-learn

---

📖 Theory

🔹 Multiple Linear Regression

$$y = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \dots + \theta_n x_n$$

• Uses multiple independent variables

• Improves prediction accuracy compared to single-variable model

---

🔹 Handling Categorical Data

The dataset contains a categorical feature:

• ocean_proximity

This must be converted to numerical form using:

• One-Hot Encoding

---

🔹 Evaluation Metrics

• MSE → Measures prediction error

• R² → Measures goodness of fit

---

📋 Procedure

1. Load dataset

2. Handle missing values

3. Encode categorical feature (ocean_proximity)

4. Separate features and target

5. Split into training and testing sets

6. Train model using LinearRegression

7. Predict values

8. Evaluate model

9. Analyze coefficients

---

💻 Program

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

# -----------------------------
# Load dataset
# -----------------------------
df = pd.read_csv("housing.csv")

# -----------------------------
# Preprocessing
# -----------------------------

# Handle missing values
df = df.dropna()

# One-hot encoding for categorical column
df = pd.get_dummies(df, columns=['ocean_proximity'], drop_first=True)

# -----------------------------
# Feature and Target Selection
# -----------------------------
X = df.drop('median_house_value', axis=1)
y = df['median_house_value']

# -----------------------------
# Train-Test Split
# -----------------------------
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# -----------------------------
# Model Training
# -----------------------------
model = LinearRegression()
model.fit(X_train, y_train)

# -----------------------------
# Model Parameters
# -----------------------------
print("Intercept:", model.intercept_)

coeff_df = pd.DataFrame({
    'Feature': X.columns,
    'Coefficient': model.coef_
})

print(coeff_df)

# -----------------------------
# Predictions
# -----------------------------
y_pred = model.predict(X_test)

# -----------------------------
# Evaluation
# -----------------------------
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

print("MSE:", mse)
print("R²:", r2)

# -----------------------------
# Visualization (Actual vs Predicted)
# -----------------------------
plt.scatter(y_test, y_pred)
plt.xlabel("Actual Values")
plt.ylabel("Predicted Values")
plt.title("Actual vs Predicted (All Features)")

plt.show()

---

📊 Output

• Intercept value

• Coefficients for all features

• MSE and R² score

• Scatter plot (Actual vs Predicted)

Intercept: -2299273.7621035315 Feature Coefficient 0 longitude -27108.746321 1 latitude -25657.807543 2 housing_median_age 1081.364206 3 total_rooms -6.322146 4 total_bedrooms 103.004042 5 population -36.409751 6 households 43.142725 7 median_income 39277.083020 8 ocean_proximity_INLAND -39240.217778 9 ocean_proximity_ISLAND 213653.374463 10 ocean_proximity_NEAR BAY -6232.416876 11 ocean_proximity_NEAR OCEAN 3166.477128 MSE: 4802173538.60416 R²: 0.6488402154431994

---

Result

The Multiple Linear Regression model was successfully implemented using all features, and performance was evaluated using MSE and R².

• Using multiple features improves prediction accuracy

• Categorical data must be encoded

• Model performance depends on feature quality