How to use ExtraTreeClassifier to predict multiclass classifications
<p>I'm quite new to machine learning techniques, and I'm having trouble following some of the scikit-learn documentation and other stackoverflow posts.. I'm trying to create a simple model from a bunch of medical data that will help me predict which of three classes a patient could fall into. </p>

<p>I load the data via pandas, convert all the objects to integers (Male = 0, Female=1 for example), and run the following code:</p>

<pre><code>import numpy as np
import pandas as pd
from sklearn.cross_validation import train_test_split
from sklearn.preprocessing import label_binarize
from sklearn.ensemble import ExtraTreesClassifier

# Upload data file with all integers:
data = pd.read_csv('datafile.csv')

y = data["Target"]
features = list(data.columns[:-1]) # Last column being the target data
x = data[features]

ydata = label_binarize(y, classes=[0, 1, 2])
n_classes = ydata.shape[1]
X_train, X_test, y_train, y_test = train_test_split(x, ydata, test_size=.5)

model2 = ExtraTreesClassifier()
model2.fit(X_train, y_train)

out = model2.predict(X_test)
print np.min(out),np.max(out)
</code></pre>

<p>The predicted values of <code>out</code> range between 0.0 and 1.0, but the classes I am trying to predict are 0,1, and 2. What am I missing?</p>

<p>That's normal behaviour in scikit-learn.</p>

<p>There are two approaches possible:</p>

<h3>A:You use "label binarize"</h3>

<ul>
<li>Binarizing transforms <code>y=[n_samples, ] -&gt; y[n_samples, n_classes]</code> (1 dimension added; integers in range(0, X) get transformed to binary values)</li>
<li>Because of this input to fit, <code>classifier.predict()</code> will also return results of the form <code>[n_predict_samples, n_classes]</code> (with 0 and 1 as the only values) / <strong>That's what you observe!</strong></li>
<li>Example output: <code>[[0 0 0 1], [1 0 0 0], [0 1 0 0]]</code> = predictions for class: 3, 0, 1</li>
</ul>

<h3>B: You skip "label binarize" (multi-class handling automatically done by sklearn)</h3>

<ul>
<li>Without binarizing (assuming your data is using integer-markers for classes): <code>y=[n_samples, ]</code></li>
<li>Because of this input to fit, <code>classifier.predict()</code> will also return results of the form <code>[n_predict_samples, ]</code> (with possibly other values than 0, 1)</li>
<li>Example output conform to above example: <code>[3 0 1]</code></li>
</ul>

<p>Both outputs are mentioned in the docs <a href="http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.ExtraTreesClassifier.html#sklearn.ensemble.ExtraTreesClassifier.predict" rel="nofollow">here</a>:</p>

<pre><code>predict(X)

Returns:    
    y : array of shape = [n_samples] or [n_samples, n_outputs]
        The predicted classes.
</code></pre>

<p><strong>Remark</strong>: the above behaviour should be valid for most/all classifiers! (not only <em>ExtraTreesClassifier</em>)</p>

python python-2.7 scikit-learn decision-tree