How to get classes labels from cross_val_predict used with predict_proba in scikit-learn
<p>I need to train a <a href="http://scikit-learn.org/dev/modules/generated/sklearn.ensemble.RandomForestClassifier.html" rel="nofollow">Random Forest classifier</a> using a 3-fold cross-validation. For each sample, I need to retrieve the prediction probability when it happens to be in the test set.</p>

<p>I am using scikit-learn version 0.18.dev0.</p>

<p>This new version adds the feature to use the method <a href="http://scikit-learn.org/dev/modules/generated/sklearn.model_selection.cross_val_predict.html#sklearn.model_selection.cross_val_predict" rel="nofollow">cross_val_predict()</a> with an additional parameter <code>method</code> to define which kind of prediction require from the estimator.</p>

<p>In my case I want to use the <a href="http://scikit-learn.org/dev/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier.predict_proba" rel="nofollow">predict_proba()</a> method, which returns the probability for each class, in a multiclass scenario.</p>

<p>However, when I run the method, I get as a result the matrix of prediction probabilities, where each rows represents a sample, and each column represents the prediction probability for a specific class.</p>

<p>The problem is that the method does not indicate which class corresponds to each column.</p>

<p>The value I need is the same (in my case using a <code>RandomForestClassifier</code>) returned in the attribute classes_ defined as:</p>

<blockquote>
  <p>classes_ : array of shape = [n_classes] or a list of such arrays
  The classes labels (single output problem), or a list of arrays of class labels (multi-output problem).</p>
</blockquote>

<p>which is needed by <code>predict_proba()</code> because in its documentation it is written that:</p>

<blockquote>
  <p>The order of the classes corresponds to that in the attribute classes_.</p>
</blockquote>

<p>A minimal example is the following:</p>

<pre><code>import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_predict

clf = RandomForestClassifier()

X = np.random.randn(10, 10)
y = y = np.array([1] * 4 + [0] * 3 + [2] * 3)

# how to get classes from here?
proba = cross_val_predict(estimator=clf, X=X, y=y, method="predict_proba")

# using the classifier without cross-validation
# it is possible to get the classes in this way:
clf.fit(X, y)
proba = clf.predict_proba(X)
classes = clf.classes_
</code></pre>

<p>Yes, they will be in sorted order; this is because <code>DecisionTreeClassifier</code> (which is the default <code>base_estimator</code> for <code>RandomForestClassifier</code>) <a href="https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/tree.py#L193" rel="nofollow">uses <code>np.unique</code> to construct the <code>classes_</code> attribute</a> which returns the sorted unique values of the input array.</p>

python scikit-learn cross-validation