Reusing the same layers for training and testing, but creating different nodes
<p>I'm trying to (re)train AlexNet (based on the code <a href="http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/" rel="nofollow">found here</a>) for a particular binary classification problem. Since my GPU is not very powerful, I settled on a batch size of 8 for training. This size determines the shape of the input tensor (8,227,227,3). However, one can use a larger batch size for the testing process, since there is no backprop involved.</p>

<p>My question is, how could I reuse the already trained hidden layers to create a different network on the same graph specifically for testing?</p>

<p>Here's a snippet of what I have tried to do:</p>

<pre><code>NUM_TRAINING_STEPS = 200
BATCH_SIZE = 1
LEARNING_RATE = 1e-1

IMAGE_SIZE = 227
NUM_CHANNELS = 3

NUM_CLASSES = 2
def main():

    graph = tf.Graph()

    trace = Tracer()

    train_data = readImage(filename1)
    test_data = readImage(filename2)
    train_labels = np.array([[0.0,1.0]])

    with graph.as_default():
        batch_data = tf.placeholder(tf.float32, shape=(BATCH_SIZE, IMAGE_SIZE, IMAGE_SIZE, NUM_CHANNELS) )
        batch_labels = tf.placeholder(tf.float32, shape=(BATCH_SIZE, NUM_CLASSES) )

        logits_training = createNetwork(batch_data)
        loss = lossLayer(logits_training, batch_labels)

        train_prediction = tf.nn.softmax(logits_training)

        print 'Prediction shape: ' + str(train_prediction.get_shape())

        optimizer = tf.train.GradientDescentOptimizer(learning_rate=LEARNING_RATE).minimize(loss) 

        test_placeholder = tf.placeholder(tf.float32, shape=(1, IMAGE_SIZE, IMAGE_SIZE, NUM_CHANNELS) )   
        logits_test = createNetwork(test_placeholder)

        test_prediction = tf.nn.softmax(logits_test)

        with tf.Session(graph=graph) as session:
            tf.initialize_all_variables().run()


            for step in range(NUM_TRAINING_STEPS):
                print 'Step #: ' + str(step+1)

                feed_dict = {batch_data: train_data, batch_labels : train_labels}

                _, l, predictions = session.run([optimizer, loss, train_prediction], feed_dict=feed_dict)

            feed_dict = {batch_data:test_data, test_placeholder:test_data}
            logits1, logits2 = session.run([logits_training,logits_test],feed_dict=feed_dict)

            print (logits1 - logits2)
return
</code></pre>

<p>I'm only training with a single image, just to evaluate whether network is actually being trained and if the values of logits1 and logits2 are the same. They are not, by several orders of magnitude.</p>

<p>createNetwork is a function which loads the weights for AlexNet and builds the model, based on the code for the myalexnet.py script found on the page to which I linked.</p>

<p>I've tried to replicate the examples from the Udacity course on Deep Learning, in particular, assignments 3 and 4.</p>

<p>If anyone could figure out how I could use the same layers for training and testing, I would be very grateful.</p>

<p>Use <code>shape=None</code>for your placeholders: <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/io_ops.html#placeholder" rel="nofollow">placeholder doc</a></p>

<p>This way you can feed any shape of data. Another (worse) option is to recreate your graph for testing with the shapes that you need, and load the ckpt that was created during training.</p>

python tensorflow