scikit-learn SVM.SVC() is extremely slow
<p>i try to use the SVM classifier to train the data which is around 100k, but I find it is extremely slow and there is no any response when I run the code after 2 hr. when dataset is around 1k, i can get the result immediately, I also try the SGDClassifier and naÃ¯ve bayes which is quite fast and get result within couple of mins.</p>

<h3>General remarks about SVM-learning</h3>

<p>SVM-training (with nonlinear-kernels; default in sklearn's SVC) is complexity-wise approximately (depends on the data and parameters) <strong>using the widely used SMO-algorithm (don't compare it with SGD-based approaches)</strong>: <code>O(n_samples^2 * n_features)</code> <a href="https://www.quora.com/What-is-the-computational-complexity-of-an-SVM" rel="nofollow">link to some question with this approximation given by one of sklearn's devs</a>.</p>

<p>So we can do some math to approximate the time-difference between 1k and 100k samples:</p>

<pre><code>1k = 1000^2 = 1.000.000 steps = Time X
100k = 100.000^2 = 10.000.000.000 steps = Time X * 10000 !!!
</code></pre>

<p>This is only an approximation and can be even worse or less worse (e.g. setting cache-size; trading-off memory for speed-gains)!</p>

<h3>Scikit-learn specific remarks</h3>

<p>The situation could also be much more complex because of all that nice stuff scikit-learn is doing for us behind the bars. The above is valid for the classic 2-class SVM. If you are by any chance trying to learn some multi-class data; scikit-learn will automatically use OneVsRest or OneVsAll approaches to do this (as the core SVM-algorithm does not support this). Read up scikit-learns docs to understand this part.</p>

<p>The same warning applies to generating probabilities: SVM's do not naturally produce probabilities for final-predictions. So to use these (activated by parameter) scikit-learn uses a heavy cross-validation procedure called <strong>Platt-scaling</strong> which will take a lot of time too!</p>

<h3>Scikit-learn documentation</h3>

<p>Because sklearn has one of the best docs, there is often a good part within these docs to explain something like that (<a href="http://scikit-learn.org/stable/modules/svm.html#complexity" rel="nofollow">link</a>):</p>

<p><a href="https://i.stack.imgur.com/gQlv9.png" rel="nofollow"><img src="https://i.stack.imgur.com/gQlv9.png" alt="enter image description here"></a></p>

python scikit-learn svm