SMS Spam Detection
An end-to-end SMS spam classifier built with classical NLP and Multinomial Naive Bayes.
- Status
- Archived
- Date
- Context
- Learning
- Language
- Python
- Stars
- 0
- Last commit
- 2026-01-30
- Code manual
- Writing assisted
This repository contains an end-to-end implementation of an SMS spam detection system built using classical Natural Language Processing (NLP) techniques and the Multinomial Naive Bayes algorithm.
The goal of this project is to understand and implement the full lifecycle of a text classification system: from raw text preprocessing to feature extraction, model training, evaluation on unseen data, and model persistence for later use.
This project is intentionally implemented without deep learning in order to focus on probabilistic modeling, interpretability, and core NLP fundamentals.
Problem Definition
Spam messages are unsolicited messages that often contain phishing attempts, financial scams, or misleading offers. From a machine learning perspective, spam detection is a binary text classification problem:
- Class 1: Spam
- Class 0: Not-Spam (Ham)
Given an SMS message, the model estimates the probability that the message belongs to each class and assigns the label with the higher posterior probability.
Dataset
The model is trained on the SMS Spam Collection Dataset, which consists of labeled SMS messages categorized as either spam or ham.
Each data point contains:
label: spam or hammessage: raw SMS text
Text Preprocessing
Since Naive Bayes relies entirely on token frequencies and probability estimates, text preprocessing directly impacts model performance. All preprocessing steps are applied consistently during both training and inference.
Preprocessing Steps
-
Lowercasing
- Normalizes word casing (e.g.,
FREE,Free,free→free) - Reduces vocabulary size
- Normalizes word casing (e.g.,
-
Selective Character Filtering
- Removes all characters except:
- lowercase letters
- whitespace
$and!
- These symbols are preserved because they often carry semantic meaning in spam messages
- Removes all characters except:
-
Tokenization
- Messages are split into individual tokens using NLTK’s tokenizer
-
Stop Word Removal
- Common English stop words are removed to reduce noise
-
Stemming
- Porter Stemmer is used to reduce tokens to their root forms
- This consolidates morphological variants and improves statistical efficiency
-
Rejoining Tokens
- Tokens are joined back into a single string to maintain compatibility with vectorizers
Feature Extraction
Raw text cannot be directly processed by machine learning models, so messages are converted into numerical feature vectors using a Bag-of-Words representation.
Method
- Vectorizer:
CountVectorizer - Feature Types:
- Unigrams (single words)
- Bigrams (pairs of consecutive words)
Including bigrams allows the model to capture limited local context (e.g., free prize) while keeping the feature space manageable.