Simple Linear Regression using Sample Data ( Single variable , Toy Example)

 

Experiment

Simple Linear Regression using Sample Data

Aim

To implement Simple Linear Regression on sample data and manually compute:

• Regression coefficients

• Mean Squared Error (MSE)

• R-squared (R²)

---

 Objectives

• Build a regression model from scratch

• Compute slope and intercept manually

• Evaluate model using MSE and R² (without sklearn)

• Visualize regression line

---

🛠️ Tools Required

• Python

• NumPy

• Matplotlib

---

📖 Theory

🔹 Linear Regression Model

$$y = b_0 + b_1 x$$

---

🔹 Regression Coefficients

$$b_1 = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sum (x - \bar{x})^2}$$ $$b_0 = \bar{y} - b_1 \bar{x}$$

---

🔹 Mean Squared Error (MSE)

$$MSE = \frac{1}{n} \sum (y_i - \hat{y}_i)^2$$

• Measures average squared error

• Lower value → better model

---

🔹 R-squared (R² Score)

$$R^2 = 1 - \frac{SS_{res}}{SS_{tot}}$$

Where:

$$SS_{res} = \sum (y_i - \hat{y}_i)^2$$
$$SS_{tot} = \sum (y_i - \bar{y})^2$$

• R² = 1 → Perfect fit

• R² = 0 → No explanatory power

---

📋 Procedure

1. Define dataset

2. Compute mean values

3. Calculate regression coefficients

4. Predict values

5. Compute MSE manually

6. Compute R² manually

7. Plot results

---

💻 Program

import numpy as np
import matplotlib.pyplot as plt

# Sample data
x = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
y = np.array([1, 3, 2, 5, 7, 8, 8, 9, 10, 12])

# Scatter plot
plt.scatter(x, y, color="m", marker="o", s=30)

# Mean values
m_x = np.mean(x)
m_y = np.mean(y)

# Calculate SS_xy and SS_xx
SS_xy = np.sum((x - m_x) * (y - m_y))
SS_xx = np.sum((x - m_x) * (x - m_x))

# Regression coefficients
b1 = SS_xy / SS_xx
b0 = m_y - b1 * m_x

print("Slope (b1):", b1)
print("Intercept (b0):", b0)

# Predictions
y_pred = b0 + b1 * x

# -----------------------------
# Manual MSE Calculation
# -----------------------------
n = len(y)
mse = np.sum((y - y_pred) ** 2) / n

print("Mean Squared Error (MSE):", mse)

# -----------------------------
# Manual R² Calculation
# -----------------------------
SS_res = np.sum((y - y_pred) ** 2)
SS_tot = np.sum((y - m_y) ** 2)

r2 = 1 - (SS_res / SS_tot)

print("R-squared (R2):", r2)

# Plot regression line
plt.plot(x, y_pred, color="g", label="Regression Line")

plt.xlabel('X')
plt.ylabel('Y')
plt.title('Linear Regression with MSE & R²')
plt.legend()

plt.show()

---

Output

• Slope and intercept values

• Mean Squared Error (MSE)

• R² score

• Graph showing regression line

Slope (b1): 1.1696969696969697 Intercept (b0): 1.2363636363636363 Mean Squared Error (MSE): 0.5624242424242423 R-squared (R2): 0.952538038613988

---

Result

The regression model was successfully implemented.
Model performance was evaluated using manually computed:

• MSE

• R² score

• MSE gives error magnitude

• R² gives goodness of fit

• Both metrics together provide better evaluation