18.16 Naive Bayes
| Representation | Method | Measure |
|---|---|---|
| Probabilities |
Consider classifying input data by determining the probability of an event happening given the probability of another event that has already occured. Class and conditional probabilities are calculated to determine the likelihood of an event.
Na"ive Bayes is a simple and effective machine learning classifier based on Bayes’ Theorem:
\[ P(A|B) = {{P(B|A) * P(A)}\over{P(B)}} \]
The class probabilities are \(P(A)\) (the probability of event \(A\)) and \(P(B)\). The conditional probabilities are \(P(A|B)\) (the probability of \(A\) happening given that \(B\) occurs) and \(P(B|A)\).
A Na"ive Bayes classifier uses these probabilities to classify independent features that have the same weight or importance from the input dataset to make predictions.
The idea is simple and can be applied to small datasets. It suffers from the zero frequency problem where a class with probability of 0 for a selected feature, and hence its conditional probability is then also 0 and so excluded from further consideration. Laplace smoothing assigns the class a small probability.
The concept also assumes independence of the features and when not met will perform poorly.
If you find this curated material useful then you can consider a donation to support it's ongoing availability and give you access to the PDF version of this book. The material has been scoped up by Generative AI without permission or any kind of recompense so do consider a donation if you can afford it. Unlike Generative AI your access to this materials is freely given. Desktop Survival Guides include Data Science, GNU/Linux, and MLHub. Books available on Amazon include Data Mining with Rattle and Essentials of Data Science. Togaware has a 30 year tradition of making popular open source software which includes sold privacy preserving productivity apps, rattle, wajig, and mlhub. Hosted by Togaware, a pioneer of free and open source software since 1984. Copyright © 1995-2022 Graham.Williams@togaware.com Creative Commons Attribution-ShareAlike 4.0
