sklearn SVM performing awfully poor
<p>I have 9164 points, where 4303 are labeled as the class I want to predict and 4861 are labeled as not that class. They are no duplicate points.</p>

<p>Following <a href="http://stackoverflow.com/questions/39500894/how-to-split-into-train-test-and-evaluation-sets-in-sklearn">How to split into train, test and evaluation sets in sklearn?</a>, and since my <code>dataset</code> is a tuple of 3 items (id, vector, label), I do:</p>

<pre><code>df = pd.DataFrame(dataset)
train, validate, test = np.split(df.sample(frac=1), [int(.6*len(df)), int(.8*len(df))])

train_labels = construct_labels(train)
train_data = construct_data(train)

test_labels = construct_labels(test)
test_data = construct_data(test)

def predict_labels(test_data, classifier):
    labels = []
    for test_d in test_data:
        labels.append(classifier.predict([test_d]))
    return np.array(labels)

def construct_labels(df):
    labels = []
    for index, row in df.iterrows():
        if row[2] == 'Trump':
            labels.append('Atomium')
        else:
            labels.append('Not Trump')
    return np.array(labels)

def construct_data(df):
    first_row = df.iloc[0]
    data = np.array([first_row[1]])
    for index, row in df.iterrows():
        if first_row[0] != row[0]:
            data = np.concatenate((data, np.array([row[1]])), axis=0)
    return data
</code></pre>

<p>and then:</p>

<pre><code>&gt;&gt;&gt; classifier = SVC(verbose=True)
&gt;&gt;&gt; classifier.fit(train_data, train_labels)
[LibSVM].......*..*
optimization finished, #iter = 9565
obj = -2718.376533, rho = 0.132062
nSV = 5497, nBSV = 2550
Total nSV = 5497
SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=True)
&gt;&gt;&gt; predicted_labels = predict_labels(test_data, classifier)
&gt;&gt;&gt; for p, t in zip(predicted_labels, test_labels):
...     if p == t:
...             correct = correct + 1
</code></pre>

<p>and I get correct only 943 labels out of 1833 (=len(test_labels)) -> (943*100/1843 = 51.4%)</p>

<hr>

<p>I am suspecting I am missing something big time here, maybe I should set a <a href="http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html" rel="nofollow">parameter</a> to the classifier to do more refined work or something?</p>

<p>Note: First time using SVMs here, so anything you might get for granted, I might have not even imagine...</p>

<hr>

<p>Attempt:</p>

<p>I went ahed and decreased the number of negative examples to 4303 (same number as positive examples). This slightly improved accuracy.</p>

<hr>

<p>Edit after the answer:</p>

<pre><code>&gt;&gt;&gt; print(clf.best_estimator_)
SVC(C=1000.0, cache_size=200, class_weight='balanced', coef0=0.0,
  decision_function_shape=None, degree=3, gamma=0.0001, kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False)
&gt;&gt;&gt; classifier = SVC(C=1000.0, cache_size=200, class_weight='balanced', coef0=0.0,
...   decision_function_shape=None, degree=3, gamma=0.0001, kernel='rbf',
...   max_iter=-1, probability=False, random_state=None, shrinking=True,
...   tol=0.001, verbose=False)
&gt;&gt;&gt; classifier.fit(train_data, train_labels)
SVC(C=1000.0, cache_size=200, class_weight='balanced', coef0=0.0,
  decision_function_shape=None, degree=3, gamma=0.0001, kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False)
</code></pre>

<p>Also I tried <code>clf.fit(train_data, train_labels)</code>, which performed the same.</p>

<hr>

<p>Edit with data (the data are not random):</p>

<pre><code>&gt;&gt;&gt; train_data[0]
array([  20.21062112,   27.924016  ,  137.13815308,  130.97432804,
        ... # there are 256 coordinates in total
         67.76352596,   56.67798138,  104.89566517,   10.02616417])
&gt;&gt;&gt; train_labels[0]
'Not Trump'
&gt;&gt;&gt; train_labels[1]
'Trump'
</code></pre>

<p>After the comments of sascha and the answer of shahins, I did this eventually:</p>

<pre><code>df = pd.DataFrame(dataset)
train, validate, test = np.split(df.sample(frac=1), [int(.6*len(df)), int(.8*len(df))])

train_labels = construct_labels(train)
train_data = construct_data(train)

test_labels = construct_labels(test)
test_data = construct_data(test)

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
train_data = scaler.fit_transform(train_data)

from sklearn.svm import SVC
# Classifier found with shahins' answer
classifier = SVC(C=10, cache_size=200, class_weight='balanced', coef0=0.0,
  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False)
classifier = classifier.fit(train_data, train_labels)

test_data = scaler.fit_transform(test_data)
predicted_labels = predict_labels(test_data, classifier)
</code></pre>

<p>and got:</p>

<pre><code>&gt;&gt;&gt; correct_labels = count_correct_labels(predicted_labels, test_labels)
&gt;&gt;&gt; print_stats(correct_labels, len(test_labels))
Correct labels = 1624
Accuracy = 88.5979268958
</code></pre>

<p>with these methods:</p>

<pre><code>def count_correct_labels(predicted_labels, test_labels):
    correct = 0
    for p, t in zip(predicted_labels, test_labels):
        if p[0] == t:
            correct = correct + 1
    return correct

def print_stats(correct_labels, len_test_labels):
    print "Correct labels = " + str(correct_labels)
    print "Accuracy = " + str((correct_labels * 100 / float(len_test_labels)))
</code></pre>

<p>I was able to optimize more with more hyper parameter tuning!</p>

<p>Helpful link: <a href="http://scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html" rel="nofollow">RBF SVM parameters</a></p>

<hr>

<p>Note: If I don't transform the <code>test_data</code>, accuracy is 52.7%.</p>

python pandas machine-learning scikit-learn classification