Is the collection in tensorflow.get_collection() cleared?
<p>I'm learning about neural nets using Tensorflow through the Stanford course. I found this while implementing a RNN and did not quite understand why the losses are accumulated:</p>

<pre class="lang-python prettyprint-override"><code># This adds a loss operation to the Graph for batch training
def add_loss_op(self, output):
    all_ones = [tf.ones([self.config.batch_size * self.config.num_steps])]
    cross_entropy = sequence_loss(
        [output], [tf.reshape(self.labels_placeholder, [-1])], all_ones, len(self.vocab))
    tf.add_to_collection('total_loss', cross_entropy)
    # Doesn't this increase in size every batch of training?
    loss = tf.add_n(tf.get_collection('total_loss')) 
    return loss
</code></pre>

<p>The documentation for <code>get_collection()</code>  <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/framework.html#Graph.get_collection" rel="nofollow">here</a> doesn't mention anything about clearing the variables. Since this is run for every training step, are the losses incremented every epoch / minibatch of training and carried over? </p>

<p>I am still new to NNs so do correct any misunderstanding I have on this!</p>

<p>I believe the add_n here is actually just to make sure that any pre-existing losses in the 'total_loss' collection get added in for the final result. It's not changing any variables, just summing up its inputs and returning the total.</p>

python neural-network tensorflow recurrent-neural-network