GridSearch with SVM producing IndexError
<p>I'm building a classifier using an SVM and want to perform a Grid Search to help automate finding the optimal model. Here's the code:</p>

<pre><code>from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.multiclass import OneVsRestClassifier

X.shape     # (22343, 323)
y.shape     # (22343, 1)

X_train, X_test, y_train, y_test = train_test_split(
  X, Y, test_size=0.4, random_state=0
)

tuned_parameters = [
  {
    'estimator__kernel': ['rbf'],
    'estimator__gamma': [1e-3, 1e-4],
    'estimator__C': [1, 10, 100, 1000]
  },
  {
    'estimator__kernel': ['linear'], 
    'estimator__C': [1, 10, 100, 1000]
  }
]

model_to_set = OneVsRestClassifier(SVC(), n_jobs=-1)
clf = GridSearchCV(model_to_set, tuned_parameters)
clf.fit(X_train, y_train)
</code></pre>

<p>and I get the following error message (this isn't the whole stack trace. just the last 3 calls): </p>

<pre><code>----------------------------------------------------
/anaconda/lib/python3.5/site-packages/sklearn/model_selection/_split.py in split(self, X, y, groups)
     88         X, y, groups = indexable(X, y, groups)
     89         indices = np.arange(_num_samples(X))
---&gt; 90         for test_index in self._iter_test_masks(X, y, groups):
     91             train_index = indices[np.logical_not(test_index)]
     92             test_index = indices[test_index]

/anaconda/lib/python3.5/site-packages/sklearn/model_selection/_split.py in _iter_test_masks(self, X, y, groups)
    606 
    607     def _iter_test_masks(self, X, y=None, groups=None):
--&gt; 608         test_folds = self._make_test_folds(X, y)
    609         for i in range(self.n_splits):
    610             yield test_folds == i

/anaconda/lib/python3.5/site-packages/sklearn/model_selection/_split.py in _make_test_folds(self, X, y, groups)
    593         for test_fold_indices, per_cls_splits in enumerate(zip(*per_cls_cvs)):
    594             for cls, (_, test_split) in zip(unique_y, per_cls_splits):
--&gt; 595                 cls_test_folds = test_folds[y == cls]
    596                 # the test split can be too big because we used
    597                 # KFold(...).split(X[:max(c, n_splits)]) when data is not 100%

IndexError: too many indices for array
</code></pre>

<p>Also, when I try reshaping the arrays so that the y is (22343,) I find that the GridSearch never finishes even if I set the tuned_parameters to only default values.</p>

<p>And here are the versions for all of the packages if that helps:</p>

<p>Python: 3.5.2</p>

<p>scikit-learn: 0.18</p>

<p>pandas: 0.19.0</p>

<p>It seems that there is no error in your implementation.</p>

<p>However, as it's mentioned in the <code>sklearn</code>documentation, the "fit time complexity is more than quadratic with the number of samples which makes it hard to scale to dataset with more than a couple of <code>10000</code> samples". <a href="http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html" rel="nofollow">See documentation here</a></p>

<p>In your case, you have <code>22343</code> samples, which can lead to some computational problems/memory issues. That is why when you do your default CV it takes a lot of time. Try to <strong>reduce</strong> your train set using <code>10000</code> samples or less.</p>

python pandas machine-learning scikit-learn svm