Save and restore in CNN
<p>i am beginner in tensorflow and i have one question related to save and restore checkpoint in Convolutional neural networks. I am trying to create CNN to classify faces. My question is:</p>

<p>If is it possible when i add new class in my dataset to make partial training? So i just want to retrain the new class instead to retrain the hole Network. is it possible to restore the weights and bias from previous training and train just the new class ?</p>

<p>i am using for save </p>

<pre><code>saver = tf.train.Saver(tf.all_variables())
save = saver.save(session, "/home/owner//tensorflownew_models.ckpt")

print("Model saved in file: %s" % save)
</code></pre>

<p>Your question has multiple facets. Let's look at each in detail:</p>

<ul>
<li><p><strong>Is it possible to restore the weights and bias from previous training?</strong> Yes. You can create a <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/state_ops.html#Saver" rel="nofollow"><code>tf.train.Saver</code></a> in your new program and use it to load the values of variables from an old checkpoint. If you only want to load some of the variables from an old model, you can specify the subset of variables that you want to restore in the <code>var_list</code> argument to the <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/state_ops.html#Saver.__init__" rel="nofollow"><code>tf.train.Saver</code> constructor</a>. If the variables in your new model have different names, you may need to specify a key-value mapping, as discussed in <a href="http://stackoverflow.com/a/39453210/3574081">this answer</a>.</p></li>
<li><p><strong>Is it possible to add a class to the network?</strong> Yes, although it's a little tricky (and there might be other ways to do it). I assume you have a softmax classifier in your network, which includes a linear layer (matrix multiplication by a weight matrix of size <code>m * C</code>, followed by adding a vector of biases of length <code>C</code>). To add a class, you could create a matrix of size <code>m * C+1</code> and a vector of length <code>C+1</code>, then initialize the first <code>C</code> rows/elements of these from the existing weights using <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/state_ops.html#Variable.scatter_assign" rel="nofollow"><code>tf.Variable.scatter_assign()</code></a>. <a href="http://stackoverflow.com/q/34913762/3574081">This question</a> deals with the same topic.</p></li>
<li><p><strong>Is it possible to do partial training?</strong> Yes. I assume you mean "training only some of the layers, while holding other layers constant." You can do this as <a href="http://stackoverflow.com/a/39450900/3574081">MMN suggests</a>  by passing an explicit list of variables to optimize when calling <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/train.html#Optimizer.minimize" rel="nofollow"><code>tf.train.Optimizer.minimize()</code></a>. For example, if you were adding a class as above, you might retrain only the softmax weights and biases, and hold the convolution layers' filters constant. Have a look at the <a href="https://www.tensorflow.org/versions/r0.10/how_tos/image_retraining/index.html" rel="nofollow">tutorial on transfer learning</a> with the pre-trained Inception model for more ideas.</p></li>
</ul>

python python-2.7 tensorflow conv-neural-network