Naive Bayesian Classifier
Description
The naive Bayesian classifier (NBS) is used in statistical research to classify a new object based on the conditional and prior probabilities trained from available data. The functionality of NBS is to produce a decision on which class an object is most likely to belong to (Augmented, 2017). NBS is based on Bayes’ mathematical theorem, which generally allows the calculation of the probability of an event given that a related other event has occurred.
Examples
To understand the functionality and practical benefits of the NBS algorithm, we can use a few examples of where it can be helpful. First, when a new email is received in email systems, the service must quickly analyze whether it is spam. Using NBS with previously generated conditional and a priori probabilities helps optimize the classification process of a new email. Second, when writing reviews, NBS can automatically categorize reviews as “good” or “bad,” using, for instance, word-frequency counts. In a more advanced sense, NBS can help diagnose diseases by entering data on a patient’s symptoms.
Coding
Using NBS can be done in RStudio, for example, with code generation as shown below:
install.packages(“e1071”)
library(e1071) #load the e1071 package containing machine learning tools, statistics, and NBS among others
data(iris) #load random preliminary data to analyze
model <- naiveBayes(Species ~., data = iris) #create NBS model
new_data <- data.frame(Sepal.Length = 5.1, Sepal.Width = 3.5, Petal.Length = 1.4, Petal.Width = 0.2)
predict(model, newdata = new_data) #create new random data and predict class membership
Since NBS does not provide the exact membership outcome but instead estimates the most likely outcome, the appropriate data scale (level) for the predicted outcome and predictor variables is typically between 0 and 1, including both ends, which is a typical interval for probability (Ray, 2023). However, the numerical probability is converted into a code value (class name) that already exists in the data.
K-Nearest Neighbors
Description
The K-Nearest Neighbors (KNN) algorithm is similar to NBS in functionality and practical application, although the KNN methodology differs. Strictly speaking, KNN is a machine learning algorithm that classifies a new object based on previously learned data, using proximity features as a criterion.
Examples
For example, suppose features exist for emails (duration, presence of graphics). In that case, and each of the existing emails belongs to a specific class (inbox, spam, work), then accepting a new email with the highlighted feature and the known distribution pattern of existing emails in the class creates an opportunity to find nearby emails with similar features and determine which class the new email should belong to. In other words, while NBS uses the idea of the highest probability that similar objects will belong to the same class, KNN uses the idea that similar objects are closer together in space.
The scales of the predictors in KNN must match those of the data on which the model was trained. In particular, if the training dataset contained information about bits of text in letters, then the predictor variable should also be quantized rather than treated as a string value. A similar scaling level for the outcome variable is used: it should be consistent with the training data.
Coding
Some of the use cases for KNN include image classification, demand prediction, and predicting, for example, a sports outcome, which in RStudio can be done as follows:
library(class) #load the class package containing the KNN model and needed for classification purposes
data(iris) #loading random preliminary data for analysis
model_knn <- knn(train = iris[, 1:4], test = new_data, cl = iris$Species, k = 3) #create the KNN model, where K is a random number
print(model_knn)
Business Applications of the Algorithms
Both algorithms can be used in business scenarios. For example, in a direct sales store, you could create a customer classifier based on their likelihood to purchase, which would drive the sales funnel. NBS can use customer-generated textual information, identify specific marker words and their frequencies, and then evaluate whether a given customer is highly likely to make a purchase using a probabilistic approach.
KNN, however, can also be used, for example, to assess customer behavior. For example, if a customer has made many purchases during the year and has consistently returned them, KNN can be used to classify the customer as risky. A separate approach is required for such customers, which would also affect decision-making.
References
Augmented AI. (2017). Naïve Bayes classifier – Fun and easy machine learning. YouTube.
Ray, S. (2023). Naive Bayes classifier explained: Applications and practice problems of naive Bayes classifier. Analytics Vidhya.