Keras - Fitting Neural Net with different Epoch number in a loop without starting over each time
<p>I am fitting a recurrent neural net in <code>python</code> using <code>keras</code> library. I am fitting the model with different <code>epoch</code> number by changing parameter <code>nb_epoch</code> in <code>Sequential.fit()</code> function. Currently I'm using <code>for</code> loop which starts over fitting each time I change <code>nb_epoch</code> which is lots of repeating work. Here is my code (the loop is in the bottom of the code, if you want to skip other parts of the code details):</p>

<pre><code>from __future__ import division
import numpy as np
import pandas
from keras.models import Sequential
from keras.layers.core import Dense, Activation, Dropout
from keras.layers.recurrent import LSTM
from sklearn.preprocessing import MinMaxScaler
from sklearn.learning_curve import learning_curve


####################################
###
### Here I do the data processing to create trainX, testX
###
####################################

#model create:
model = Sequential()

#this is the epoch array for different nb_epoch


####################################
###
### Here I define model architecture
###
####################################

model.compile(loss="mse", optimizer="rmsprop")


#################################################
####  Defining arrays for different epoch number 
#################################################
epoch_array = range(100, 2100,100)


# I create the following arrays/matrices to store the result of NN fit 
# different epoch number.

train_DBN_fitted_Y = np.zeros(shape=(len(epoch_array),trainX.shape[0]))
test_DBN_fitted_Y = np.zeros(shape=(len(epoch_array),testX.shape[0]))

###############################################
###
### Following loop is the heart of the question
###
##############################################

i = 0  
for epoch in epoch_array:
      model.fit( trainX, trainY,
            batch_size = 16, nb_epoch = epoch, validation_split = 0.05, verbose = 2)
      trainPredict = model.predict(trainX)
      testPredict = model.predict(testX)
      trainPredict = trainPredict.reshape(trainPredict.shape[0])
      testPredict = testPredict.reshape(testPredict.shape[0])
      train_DBN_fitted_Y[i] = trainPredict
      test_DBN_fitted_Y[i]  = testPredict
      i = i + 1
</code></pre>

<p>Now this loops is very inefficient. Because for example, when it sets, say,  <code>nb_epoch</code> = 100, it starts training from <code>epoch = 1</code> and finishes at <code>epoch = 100</code> like following : </p>

<pre><code>Epoch 1/100
0s - loss: 1.9508 - val_loss: 296.7801
.
.
.
Epoch 100/100
0s - loss: 7.6575 - val_loss: 366.2218
</code></pre>

<p>In the next iteration of loop, where it says <code>nb_epoch = 200</code> it starts training from <code>epoch = 1</code> again and finishes at <code>epoch = 200</code>. But what I want to do is, in this iteration, start training from where it left in the last iteration of the loop i.e. <code>epoch = 100</code> and then <code>epoch = 101</code> and so on....</p>

<p>How can I modify this loop to achieve this? </p>

<p>Continuously calling <code>fit</code> is training your model further, starting from the state it was left from the previous call. For it to <strong>not</strong> continue it would have to reset the weights of your model, which <code>fit</code> does not do. You are just not seeing that it does that, as it is always starting to count epochs beginning at 1.</p>

<p>So in the end the problem is just that it does not print the correct number of epochs (which you cannot change).</p>

<p>If this is bothering you, you can implement your own <code>fit</code> by calling <a href="https://keras.io/models/model/#train_on_batch" rel="nofollow"><code>model.train_on_batch</code></a> periodically.</p>

python loops neural-network deep-learning keras