Scikit Learn Categorical data with random forests
<p>I am trying to work with the titanic survival challenge in kaggle <a href="https://www.kaggle.com/c/titanic" rel="nofollow">https://www.kaggle.com/c/titanic</a>.</p>

<p>I am not experienced in R so i am using Python and Scikit Learn for the <strong>Random Forest Classifier</strong></p>

<p>I am seeing many people using scikit learn converting their categorical of many levels into dummy variables.</p>

<p>I don't understand the point of doing this, why can't we just map the levels into a numeric value and be done with it.</p>

<p>And  also i saw someone do the following:
There was a categorical feature <strong>Pclass</strong> with three levels, he created 3  dummy variables for this and dropped the variable which had the least survival rate. I couldn't understand this either, i though decision trees didn't care about correlated features.</p>

<p>If you just map levels to numeric values, python will treat your values as numeric. That is, numerically <code>1&lt;2</code> and so on even if your levels were initially unordered. Think about the "distance" problem. This distance between 1 and 2 is 1, between 1 and 3 is 2. But what were the original distances between your categorical variables? For example, what are the distances between "banana" "peach" and "apple"?  Do you suppose that they are all equal? </p>

<p>About dummy variable: if you have 3 classes and create 3 dummy variables, they not just correlated, they are linearly dependent. This is never good.</p>

python scikit-learn random-forest