Neural network accuracy optimization
<p>I have constructed an ANN in keras which has 1 input layer(3 inputs), one output layer (1 output) and two hidden layers with with 12 and 3 nodes respectively.</p>

<p><a href="http://i.stack.imgur.com/PkYfu.png" rel="nofollow"><img src="http://i.stack.imgur.com/PkYfu.png" alt="enter image description here"></a></p>

<p>The way i construct and train my network is:</p>

<pre><code>from keras.models import Sequential
from keras.layers import Dense
from sklearn.cross_validation import train_test_split
import numpy
# fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)

dataset = numpy.loadtxt("sorted output.csv", delimiter=",")
# split into input (X) and output (Y) variables
X = dataset[:,0:3]
Y = dataset[:,3]
# split into 67% for train and 33% for test
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.33, random_state=seed)
# create model
model = Sequential()
model.add(Dense(12, input_dim=3, init='uniform', activation='relu'))
model.add(Dense(3, init='uniform', activation='relu'))
model.add(Dense(1, init='uniform', activation='sigmoid'))
# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(X_train, y_train, validation_data=(X_test,y_test), nb_epoch=150, batch_size=10)
</code></pre>

<p>Sorted output csv file looks like:</p>

<p><a href="http://i.stack.imgur.com/CIKn3.png" rel="nofollow"><img src="http://i.stack.imgur.com/CIKn3.png" alt="enter image description here"></a></p>

<p>so after 150 epochs i get: <strong>loss: 0.6932 - acc: 0.5000 - val_loss: 0.6970 - val_acc: 0.1429</strong></p>

<p>My question is: how could i modify my NN in order to achieve higher accuracy?</p>

<p>Neil Slater already provided a long list of helpful general advices.</p>

<p>In your specific examaple, normalization is the important thing. If you add the following lines to your code</p>

<pre><code>...
X = dataset[:,0:3]
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X = scaler.fit_transform(X)
</code></pre>

<p>you will get 100% accuracy on your toy data, even with much simpler network structures. Without normalization, the optimizer won't work.</p>

python neural-network theano keras