SMS Spam Detection

An end-to-end SMS spam classifier built with classical NLP and Multinomial Naive Bayes.

Status
Archived
Date
Context
Learning
Language
Python
Stars
0
Last commit
2026-01-30
  • Code manual
  • Writing assisted
What do these mean?

This repository contains an end-to-end implementation of an SMS spam detection system built using classical Natural Language Processing (NLP) techniques and the Multinomial Naive Bayes algorithm.

The goal of this project is to understand and implement the full lifecycle of a text classification system: from raw text preprocessing to feature extraction, model training, evaluation on unseen data, and model persistence for later use.

This project is intentionally implemented without deep learning in order to focus on probabilistic modeling, interpretability, and core NLP fundamentals.


Problem Definition

Spam messages are unsolicited messages that often contain phishing attempts, financial scams, or misleading offers. From a machine learning perspective, spam detection is a binary text classification problem:

  • Class 1: Spam
  • Class 0: Not-Spam (Ham)

Given an SMS message, the model estimates the probability that the message belongs to each class and assigns the label with the higher posterior probability.


Dataset

The model is trained on the SMS Spam Collection Dataset, which consists of labeled SMS messages categorized as either spam or ham.

Each data point contains:

  • label: spam or ham
  • message: raw SMS text

Text Preprocessing

Since Naive Bayes relies entirely on token frequencies and probability estimates, text preprocessing directly impacts model performance. All preprocessing steps are applied consistently during both training and inference.

Preprocessing Steps

  1. Lowercasing

    • Normalizes word casing (e.g., FREE, Free, free → free)
    • Reduces vocabulary size
  2. Selective Character Filtering

    • Removes all characters except:
      • lowercase letters
      • whitespace
      • $ and !
    • These symbols are preserved because they often carry semantic meaning in spam messages
  3. Tokenization

    • Messages are split into individual tokens using NLTK’s tokenizer
  4. Stop Word Removal

    • Common English stop words are removed to reduce noise
  5. Stemming

    • Porter Stemmer is used to reduce tokens to their root forms
    • This consolidates morphological variants and improves statistical efficiency
  6. Rejoining Tokens

    • Tokens are joined back into a single string to maintain compatibility with vectorizers

Feature Extraction

Raw text cannot be directly processed by machine learning models, so messages are converted into numerical feature vectors using a Bag-of-Words representation.

Method

  • Vectorizer: CountVectorizer
  • Feature Types:
    • Unigrams (single words)
    • Bigrams (pairs of consecutive words)

Including bigrams allows the model to capture limited local context (e.g., free prize) while keeping the feature space manageable.

Vectorizer Configuration