Tensorflow Data Input Toggle: Train/Validation
<p>I have data that comes into my graph through queue runners, after I switched from the handy but speed-inferior placeholders.</p>

<p>After each training epoch, I wish to run a validation pass. Other than the the training pass, the validation pass uses different data, no augmentation and no shuffling.</p>

<p>The question is simple: how do I toggle these things?</p>

<p>A few observations:</p>

<ul>
<li>I cannot toggle the <code>shuffle</code> option in the <code>string_input_producer</code> through a <code>tf.placeholder</code> boolean.</li>
<li>The only examples on-the-line that I have found use the <code>placeholder</code> to seperate the training from the validation data. These in turn, do not use the superior queue runners.</li>
<li>I did manage to do the above with a <code>tf.cond()</code> here i would test for a <code>is_training</code> <code>tf.placeholder</code> boolean that i pass through the <code>feed_dict</code>. Is this solution the most optimal? How expensive is this <code>tf.conf()</code> method?</li>
</ul>

<p>One probable answer is usage of <code>make_template</code>
This is outlined in <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/kernel_tests/template_test.py" rel="nofollow">https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/kernel_tests/template_test.py</a> ; It basically says one could do this:</p>

<pre class="lang-py prettyprint-override"><code>training_input, training_output = ([1., 2., 3., 4.], [2.8, 5.1, 7.2, 8.7])
test_input, test_output = ([5., 6., 7., 8.], [11, 13, 15, 17])

tf.set_random_seed(1234)

def test_line(x):
  m = tf.get_variable("w", shape=[],
                      initializer=tf.truncated_normal_initializer())
  b = tf.get_variable("b", shape=[],
                      initializer=tf.truncated_normal_initializer())
  return x * m + b

line_template = template.make_template("line", test_line)

train_prediction = line_template(training_input)
test_prediction = line_template(test_input)

train_loss = tf.reduce_mean(tf.square(train_prediction - training_output))
test_loss = tf.reduce_mean(tf.square(test_prediction - test_output))

optimizer = tf.train.GradientDescentOptimizer(0.1)
train_op = optimizer.minimize(train_loss)

with tf.Session() as sess:
  sess.run(tf.initialize_all_variables())
  initial_test_loss = sess.run(test_loss)
  sess.run(train_op)
  final_test_loss = sess.run(test_loss)

# Parameters are tied, so the loss should have gone down when we trained it.
self.assertLess(final_test_loss, initial_test_loss)
</code></pre>

python tensorflow