Hybrid Naive Bayes: How to train Naive Bayes Classifer with numeric and category variable together(sklearn)
<p>Basicly, skelarn have naive bayes with Gaussian kernal which can class numeric variables.</p>

<p>However, how to deal with data set containing numeric variables and category variables together.</p>

<p>For an example, give a dataset below, how use sklearn train mixed data type together without discreting numeric variables?</p>

<p>Index  Gender  Age  Product_Reviews</p>

<p>A      Female       20   Good</p>

<p>B      Male          21   Bad</p>

<p>C      Female  25   Bad</p>

<p>I mean , for Bayes classication, P(A|B)= P(B|A)*P(A)/P(B) .</p>

<p>For category variables, P(B|A) is easily to count out, 
but for numeric variables, it should follows Gaussian distribution.
And assume we have got P(B|A) with Gaussian distribution.</p>

<p>Is there any package can directly work with these together?</p>

<p>Please be note: this question is not duplicated with <a href="http://stackoverflow.com/questions/38621053/how-can-i-use-sklearn-naive-bayes-with-multiple-categorical-features">How can I use sklearn.naive_bayes with (multiple) categorical features?</a> 
and <a href="http://stackoverflow.com/questions/14254203/mixing-categorial-and-continuous-data-in-naive-bayes-classifier-using-scikit-lea?rq=1">Mixing categorial and continuous data in Naive Bayes classifier using scikit-learn</a></p>

<p>Because this question is not wanna to do a naive bayes with dummy variables(1st question) and also do not wanna do a model ensemble(2nd question solution2).</p>

<p>The mathematic algothrim is here <a href="https://tom.host.cs.st-andrews.ac.uk/ID5059/L15-HsuPaper.pdf" rel="nofollow">https://tom.host.cs.st-andrews.ac.uk/ID5059/L15-HsuPaper.pdf</a> , which calculates conditional probabilities with Gaussian distribution instead of counting number with numeric variables. And make classification with all conditional probabilities including category variables(by counting number) and numeric variables(Gaussian distribution)</p>

<p>The answer comes directly from the mathematics of Naive Bayes</p>

<ol>
<li><p>Categorical variables provide you with log P(a|cat) ~ SUM_i log P(cat_i|a) + log P(a) (I am omitting division by P(cat), as what NB implementation returns is also ignoring it)</p></li>
<li><p>Continuous variables give you the same thing, log P(a|con) ~ SUM_i log P(con_i|a) + log P(a) (I am omitting division by P(cat), as what NB implementation returns is also ignoring it)</p></li>
</ol>

<p>and since in Naive Bayes features are independent we get that for <code>x</code> which contains both categorical and continuous</p>

<p>P(a|x) ~ SUM_i log(x_i | a) + log P(a) = SUM_i log P(cat_i|a) + log P(a) +  SUM_i log P(con_i|a) + log P(a) - log P(a) = log likelihood from categorical model + log likelihood from continuous model - log prior of class a</p>

<p>all these elements you can read out from your two models, independently fitted to each part of the data. Notice that this <strong>is not an ensemble</strong>, you simply fit two models and construct one on your own <strong>due to specific assumptions of naive bayes</strong>, thus you are overcoming implementational limitation this way, yet still efficiently constructing valid NB model on mixed distributions. Note that this works for <strong>any set of mixed distributions</strong>, thus you could do the same given more different NBs (using different distributions).</p>

python statistics scikit-learn naivebayes