Accuracy not high enough for dogs_cats classification dataset using CNN with Keras-Tf python
<p>Guys I'm trying to classify the Dogs vs Cats dataset using CNN. I'm deep learning beginner btw.</p>

<p>The dataset link can be obtained from <a href="https://www.kaggle.com/c/dogs-vs-cats/data" rel="nofollow">here</a>. I've also classified the above dataset using MLP with a training accuracy of 70% and testing accuracy of 62%. So I decided to use CNN to improve the score.</p>

<p>But unfortunately, I'm still getting very similar results. Here is my code:</p>

<pre><code>from sklearn.cross_validation import train_test_split
from sklearn.preprocessing import LabelEncoder
from keras.layers import Dense, Activation, Flatten, Dropout
from keras.layers.convolutional import Convolution2D
from keras.layers.convolutional import MaxPooling2D
from keras.models import Sequential
from keras.utils import np_utils
from keras.optimizers import SGD
from keras.datasets import mnist
from keras import backend as K
from imutils import paths
import numpy as np
import argparse
import cPickle
import h5py
import sys
import cv2
import os

K.set_image_dim_ordering('th')

def image_to_feature_vector(image, size=(28, 28)):
    return cv2.resize(image, size)

print("[INFO] pre-processing images...")
imagePaths = list(paths.list_images(raw_input('path to dataset: ')))

data   = []
labels = []

for (i, imagePath) in enumerate(imagePaths):
    image = cv2.imread(imagePath)
    label = imagePath.split(os.path.sep)[-1].split(".")[0]
    features = image_to_feature_vector(image)
    data.append(features)
    labels.append(label)

    if i &gt; 0 and i % 1000 == 0:
        print("[INFO] processed {}/{}".format(i, len(imagePaths)))

le     = LabelEncoder()
labels = le.fit_transform(labels)
labels = np_utils.to_categorical(labels, 2)
data   = np.array(data) / 255.0

print("[INFO] constructing training/testing split...")
(X_train, X_test, y_train, y_test) = train_test_split(data, labels, test_size=0.25, random_state=42)

X_train = X_train.reshape(X_train.shape[0], 3, 28, 28).astype('float32')
X_test  = X_test.reshape(X_test.shape[0], 3, 28, 28).astype('float32')
num_classes = y_test.shape[1]

def basic_model():
    model = Sequential()
    model.add(Convolution2D(32, 3, 3, border_mode='valid', init='uniform', bias=True, input_shape=(3, 28, 28), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))
    model.add(Flatten())
    model.add(Dense(128, activation='relu'))
    model.add(Dense(num_classes, activation='softmax'))

    sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
    model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
    return model

model = basic_model()

model.fit(X_train, y_train, validation_data=(X_test, y_test), nb_epoch=25, batch_size=50, shuffle=True, verbose=1)

print('[INFO] Evaluating the model on test data...')
scores = model.evaluate(X_test, y_test, batch_size=100, verbose=1)
print("\nAccuracy: %.4f%%\n\n"%(scores[1]*100))
</code></pre>

<p>The CNN model I've used is very basic but decent enough I think. I followed various tutorials to get to it. I even used this architecture but got similar result(65% testing accuracy):</p>

<pre><code>def baseline_model():
    model = Sequential()
    model.add(Convolution2D(30, 5, 5, border_mode='valid', input_shape=(3, 28, 28), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Convolution2D(15, 3, 3, activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.2))
    model.add(Flatten())
    model.add(Dense(128, activation='relu'))
    model.add(Dense(50, activation='relu'))
    model.add(Dense(num_classes, activation='softmax'))

    sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
    model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])
    return model
</code></pre>

<p>For optimiser I also tried <code>adam</code> with default parameters and for <code>model.complie</code> loss function I also tried <code>categorical_crossentropy</code> but there was no (or very slight) improvement.</p>

<p>Can you suggest where I'm going wrong or what I can do to improve efficiency?(In few epochs if possible)</p>

<p>(I'm a beginner in deep learning and keras programming...)</p>

<p>EDIT: so I managed to touch 70.224% testing accuracy and 74.27% training accuracy. CNN architecture was
<code>CONV =&gt; CONV =&gt; POOL =&gt; DROPOUT =&gt; FLATTEN =&gt; DENSE*3</code></p>

<p>(There is almost no overfitting as training acc: 74% and testing is: 70% í ½í¹)</p>

<p>But still open to suggestions to increase it further, 70% is definitely on lower side...</p>

<p>Basically, your network is not deep enough. That is why both your training and validation accuracy are low. You can try to deepen your network from two aspects.</p>

<ol>
<li><p>Use larger number of filters for each convolutional layer. (30, 5, 5) or (15, 3, 3) are just not enough. Change the first convolutional layer to (64, 3, 3). After max pooling, which reduces your 2D size, the network should provide "deeper" features. Thus, the second should not be 15, but something like (64, 3,3) or even (128,3,3).</p></li>
<li><p>Add more convolutional layers. 5 or 6 layers for this problem may be good.</p></li>
</ol>

<p>Overall, your question is beyond programming. It is more about CNN network architecture. You may read more research papers on this topic to get a better understanding. For this specific problem, Keras has a very good tutorial on how to improve the performance with very small set of cats and dogs images:
<a href="https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html" rel="nofollow">Building powerful image classification models using very little data</a></p>

python keras conv-neural-network