SVM Scikit-Learn : Why prediction time decrease with SVC when increasing parameter C?
<p>Iâm trying to evaluate the influence of the # of features &amp; parameter C (SVM regularization) on the prediction time. I am using a modified version of <a href="http://scikit-learn.org/stable/auto_examples/applications/plot_prediction_latency.html#example-applications-plot-prediction-latency-py" rel="nofollow">code</a> proposed by scikit-learn website. </p>

<p>Here are some key lines of code :
input </p>

<pre><code>'n_train': int(2000),
'n_test':  int( 500),
'n_features': np.arange(10,100,10)
</code></pre>

<p>Functions</p>

<pre><code>SVC(kernel='linear', C=0.001)
SVC(kernel='linear', C=0.01)
SVC(kernel='linear', C=1)
SVC(kernel='linear', C=100)
</code></pre>

<p>predictions</p>

<pre><code>estimator.fit(X_train, y_train)
....
start = time.time()
estimator.predict(X_test)
runtimes[i] = time.time() - start
</code></pre>

<p>Output : <a href="http://i.stack.imgur.com/I3nea.png" rel="nofollow">Evolution of Prediction Time</a>
<a href="http://i.stack.imgur.com/I3nea.png" rel="nofollow"><img src="http://i.stack.imgur.com/I3nea.png" alt="enter image description here"></a>
I donât understand why the prediction time is reversed. According to many resources (<a href="http://stats.stackexchange.com/questions/31066/what-is-the-influence-of-c-in-svms-with-linear-kernel">3</a> and others), the latency should increase with C parameter of SVM function.</p>

<p>Having a larger C will lead to smaller values for the slack variables. This means that the number of support vectors will decrease. When you run the prediction, it will need to calculate the indicator function for each support vector. </p>

<p>Thus; smaller C -> more support vectors -> more calculations -> slower predictions.</p>

python machine-learning scikit-learn svm latency