Cross validation with particular dataset lists with Python
<p>I know sklearn has nice method to get cross validation scores:</p>

<pre><code> from sklearn.model_selection import cross_val_score
 clf = svm.SVC(kernel='linear', C=1)
 scores = cross_val_score(clf, iris.data, iris.target, cv=5)
 scores      
</code></pre>

<p>I'd like to know scores with specific training and test set:</p>

<pre><code>train_list = [train1, train2, train3] # train1,2,3 is the training data sets
test_list = [test1, test2, test3] # # test1,2,3 is the test data sets
clf = svm.SVC(kernel='linear', C=1)
scores = some_nice_method(clf, train_list, test_list)
</code></pre>

<p>Is there such kind of method giving scores of particular separated data set in python?</p>

<p>My suggestion is to use <a href="http://scikit-learn.org/0.17/modules/generated/sklearn.cross_validation.KFold.html" rel="nofollow">kfold cross validation</a> like below. In this case, you will get both train, test indices for a particular split along with the accuracy score.</p>

<pre><code>from sklearn import svm
from sklearn import datasets
from sklearn.cross_validation import KFold
from sklearn.metrics import accuracy_score

iris = datasets.load_iris()
X = iris.data
y = iris.target

clf = svm.SVC(kernel='linear', C=1)
kf = KFold(len(y), n_folds=5)

for train_index, test_index in kf:
    print("TRAIN:", train_index, "TEST:", test_index)
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]
    clf.fit(X_train, y_train)
    y_pred = clf.predict(X_test)
    score = accuracy_score(y_test, y_pred)
    print score
</code></pre>

python machine-learning scikit-learn