How to use cross_val_score with random_state
<p>I'm getting different values for different runs ... what am I doing wrong here:</p>

<pre><code>X=np.random.random((100,5))
y=np.random.randint(0,2,(100,))
clf=RandomForestClassifier()
cv = StratifiedKFold(y, random_state=1)
s = cross_val_score(clf, X,y,scoring='roc_auc', cv=cv)
print(s)
# [ 0.42321429  0.44360902  0.34398496]
s = cross_val_score(clf, X,y,scoring='roc_auc', cv=cv)
print(s)
# [ 0.42678571  0.46804511  0.36090226]
</code></pre>

<p>The mistake you are making is calling the <a href="http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html" rel="nofollow"><code>RandomForestClassifier</code></a> whose default arg, <code>random_state</code> is None. So, it picks up the seed generated by <code>np.random</code> to produce the random output.</p>

<p>The <code>random_state</code> in both <a href="http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedKFold.html#sklearn.model_selection.StratifiedKFold" rel="nofollow"><code>StratifiedKFold</code></a> and <code>RandomForestClassifier</code> need to be the same inorder to produce equal arrays of scores of cross validation.</p>

<p><strong>Illustration:</strong></p>

<pre><code>X=np.random.random((100,5))
y=np.random.randint(0,2,(100,))

clf = RandomForestClassifier(random_state=1)
cv = StratifiedKFold(y, random_state=1)        # Setting random_state is not necessary here
s = cross_val_score(clf, X,y,scoring='roc_auc', cv=cv)
print(s)
##[ 0.57612457  0.29044118  0.30514706]
print(s)
##[ 0.57612457  0.29044118  0.30514706]
</code></pre>

<p>Another way of countering it would be to not provide <code>random_state</code> args for both RFC and SKF. But, simply providing the <code>np.random.seed(value)</code> to create the random integers at the beginning. These would also create equal arrays at the output.</p>

python machine-learning scikit-learn