Comparison of Multinomial and Bernoulli Naive Bayes on Sample Data

 

Experiment

Title

Comparison of Multinomial and Bernoulli Naive Bayes on Sample Dataset

---

🎯 Objective

• To implement:
  • Multinomial Naive Bayes
  • Bernoulli Naive Bayes
• To compare predictions and understand differences

🧠Theory

Naive Bayes Classifier

Naive Bayes is a probabilistic machine learning algorithm based on Bayes theorem. It is mainly used for classification problems.

The Naive Bayes classifier assumes that all features are conditionally independent.

---

Bernoulli Naive Bayes

Bernoulli Naive Bayes works with binary features.

It checks only whether a word is present or absent in a document.

Example:

Word	Present?
free	1
money	1
offer	0

It is suitable for:

• Binary feature datasets

• Short text classification

• Spam filtering

---

Multinomial Naive Bayes

Multinomial Naive Bayes works with word frequencies.

It considers how many times a word appears in a document.

Example:

Word	Count
free	3
money	1
offer	0

It is suitable for:

• Text classification

• Document classification

• Sentiment analysis

---

🔍 Difference Between Bernoulli and Multinomial Naive Bayes

Feature	Bernoulli NB	Multinomial NB
Feature Type	Binary	Frequency Count
Uses Word Frequency	No	Yes
Considers Absence of Words	Yes	No
Suitable For	Short text	General NLP tasks

---

Algorithm

Bernoulli Naive Bayes

1. Convert text into binary feature vectors.

2. Compute prior probabilities.

3. Compute conditional probabilities.

4. Apply Bayes theorem.

5. Predict the class with maximum posterior probability.

---

Multinomial Naive Bayes

1. Convert text into frequency count vectors.

2. Compute prior probabilities.

3. Compute likelihood probabilities.

4. Apply Bayes theorem.

5. Predict the class with maximum posterior probability.

---

💻 Program

from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import BernoulliNB, MultinomialNB

# -----------------------------
# 1. Sample Dataset
# -----------------------------
texts = [
    "free money now",
    "win money win prize",
    "limited time offer",
    "project meeting tomorrow",
    "schedule the meeting",
    "let us discuss project"
]

labels = [
    "Spam",
    "Spam",
    "Spam",
    "Not Spam",
    "Not Spam",
    "Not Spam"
]

# -----------------------------
# 2. Feature Extraction
# -----------------------------
vectorizer = CountVectorizer()

X = vectorizer.fit_transform(texts)

print("Vocabulary:")
print(vectorizer.get_feature_names_out())

print("\nFeature Matrix:")
print(X.toarray())

# -----------------------------
# 3. Train Models
# -----------------------------
bernoulli_model = BernoulliNB()
multinomial_model = MultinomialNB()

bernoulli_model.fit(X, labels)
multinomial_model.fit(X, labels)

# -----------------------------
# 4. Test Sample
# -----------------------------
test_text = ["free prize money"]

X_test = vectorizer.transform(test_text)

# -----------------------------
# 5. Predictions
# -----------------------------
bern_pred = bernoulli_model.predict(X_test)
multi_pred = multinomial_model.predict(X_test)

print("\nTest Text:", test_text[0])

print("\nBernoulli NB Prediction:")
print(bern_pred[0])

print("\nMultinomial NB Prediction:")
print(multi_pred[0])

# -----------------------------
# 6. Probabilities
# -----------------------------
print("\nBernoulli Probabilities:")
print(bernoulli_model.predict_proba(X_test))

print("\nMultinomial Probabilities:")
print(multinomial_model.predict_proba(X_test))

---

📊Sample Output

Vocabulary:
['discuss' 'free' 'let' 'limited' 'meeting' 'money' 'now'
 'offer' 'prize' 'project' 'schedule' 'the' 'time'
 'tomorrow' 'us' 'win']

Feature Matrix:
[[0 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0]
 [0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 2]
 [0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0]
 [0 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0]
 [0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 0]
 [1 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0]]

Test Text: free prize money

Bernoulli NB Prediction:
Spam

Multinomial NB Prediction:
Spam

Bernoulli Probabilities:
[[0.08 0.92]]

Multinomial Probabilities:
[[0.02 0.98]]

---

Result

Bernoulli Naive Bayes and Multinomial Naive Bayes classifiers were implemented successfully using a sample text dataset.

Both classifiers predicted the test message as Spam.

Bernoulli Naive Bayes considered only the presence or absence of words, whereas Multinomial Naive Bayes considered word frequencies. Therefore, Multinomial Naive Bayes produced stronger confidence for repeated spam-related words.

• Bernoulli Naive Bayes is suitable for binary feature representation.

• Multinomial Naive Bayes is suitable for text classification with word frequency information.

• Multinomial Naive Bayes generally performs better for NLP tasks because it captures repeated word import

---

📌Insight

👉 This dataset is small, so both models give similar results
👉 Differences become clearer with:

• Larger datasets
• Text classification problems