Adapting Tensorflow CIFAR10 model for own data
<p>I am just starting with tensorflow and I thought a good first step would be to adapt CIFAR10 model for my own use. My database are not images but signals and a whole database has a shape of <code>[16400,3000,1,1]</code> (dimensionwise: number of all samples, height, width and number of channels added on purpose). I am already working on this problem with MatConvNet toolbox, so this question is strictly about tensorflow machnism. The database is a ready numpy tensor of the size above, in the code below is my attempt to prepare the data to be readable for the training script</p>

<pre><code>from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import os

from six.moves import xrange  # pylint: disable=redefined-builtin
import tensorflow as tf
import numpy as np

IMAGE_SIZE = 3000

data = np.load('/home/tensorflow-master/tensorflow/models/image/cifar10 /konsensop/data.npy')
labels = np.load('/home/tensorflow-master/tensorflow/models/image/cifar10/konsensop/labels.npy')
labels = labels-1
labels = labels.astype(int)
data = tf.cast(data,tf.float32)
labels = tf.cast(labels,tf.int64)

NUM_CLASSES = 2
NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN = 10000
NUM_EXAMPLES_PER_EPOCH_FOR_EVAL = 6400
def _generate_image_and_label_batch(data_sample, label, in_queue_examples,
                                batch_size, shuffle):
num_preprocess_threads = 16
 if shuffle:
  data, label_batch = tf.train.shuffle_batch(
    [data_sample, label],
    batch_size=batch_size,
    num_threads=num_preprocess_threads,
    capacity=min_queue_examples + batch_size,
    min_after_dequeue=min_queue_examples)
 else:
  data, label_batch = tf.train.batch(
    [data_sample, label],
    batch_size=batch_size,
    num_threads=num_preprocess_threads,
    capacity=min_queue_examples +  batch_size)

  return data, tf.reshape(label_batch, [batch_size])

def inputs(data,labels, batch_size):
  for i in xrange(0, data.shape[0]/batch_size):
    data_sample = data[i,:,:,:]
    label = labels[i,0]
    height = 3000
    width = 1
    min_fraction_of_examples_in_queue = 0.4
    min_queue_examples = int(NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN*
         min_fraction_of_examples_in_queue)
    print('Filling queue with %d data before starting to train' %  min_queue_examples)
    return _generate_image_and_label_batch(data_sample, label,
                                     min_queue_examples, batch_size,
                                     shuffle=True)
</code></pre>

<p>I'm trying to load the data I aleady have and generate batches in a way cifar10 model did, but when running the trainer code I get an error in<code>data,labels = konsensop_input.inputs(data,labels,batch_size)    UnboundcocalError: local variable 'data' referenced before assigment</code> </p>

<pre><code>data = konsensop_input.data
labels = konsensop_input.labels
def train():
  with tf.Graph().as_default():
    global_step = tf.Variable(0, trainable = False)
    data, labels = konsensop_input.inputs(data, labels, batch_size)
    logits = konsensop_train.inference(data)
# calculate loss
    loss = konsensop.loss(logits, labels)
    train_op = konsensop.train(loss, global_step)
# create a saver
    saver = tf.train.Saver(tf.all_variables()) #saves all variables in a graph
# build the summary operation based on the TF collection of summaries
    summary_op = tf.merge_all_summaries()
# build an initialization operation to run below
    init  = tf.initialize_all_variables()
# start running operations on the graph
    sess = tf.Session(config = tf.ConfigProto(log_device_placement=False))
    sess.run(init)
# start the queue runners
    tf.train.start_queue_runners(sess = sess) #co to i po co to"""
    summary_writer = tf.train.SummaryWriter( FLAGS.train_dir, sess.graph)

  for step in xrange(FLAGS.max_step):
    start_time = time.time()
    _, loss_value = sess.run([train_op, loss])
    duration = time.time() - start_time
    assert not np.isnan(loss_value), 'Model diverged with loss = NaN'

    if step % 10 == 0:
      num_examples_per_step = FLAGS.batch_size
      examples_per_sec = num_examples_per_step / duration
      sec_per_batch = float(duration)

      format_str = ('%s: step %d, loss = %.2f (%.1f examples/sec; %.3f    sec/batch)')
      print ( format_str % (datetime.now(), step, loss_value, examples_per_sec, sec_per_batch))

    if step % 100 == 0:
      summary_str = sess.run(summary_op)
      summary_writer.add_summary(summary_str, step)

    if step % 1000 == 0 or (step + 1) == FLAGS.max_steps:
      checkpoint_path = os.path.join(FLAGS.train_dir, 'model.ckpt')
      saver.save(sess, checkpoint_path, global_step = step)

def main(argv=None):
 train()

if __name__=='__main__':
 tf.app.run()
</code></pre>

<p>I would like to figure out how to implement a reasonable data feeding technique here</p>

<p>For the relatively small data set you want to work with, you might consider just loading it into a big numpy array, then iterating over it in mini-batches, which you feed to the computation graph via <code>tf.placeholder</code>s and the <code>feed_dict</code> mechanism.</p>

<p>The mini-batch iteration could look something like this (you should probably add random shuffling after each epoch):</p>

<pre><code>def iterate_batches(X, y, batch_size, num_epochs):
    N = np.size(X, 0)
    batches_per_epoch = N/float(batch_size)
    for i in range(num_epochs):
        for j in range(batches_per_epoch):
            start, stop = j*batch_size, (j+1)*batch_size
            yield X[start:stop, :], y[start:stop]
</code></pre>

<p>(If you are not familiar with Python's <code>yield</code> mechanism, google for Python generators. There a lots of good introductions on the web.)</p>

<p>Given that you have a mechanism to load the whole data set into a numpy array <code>X_train, y_train</code>, you can then write your training loop like this</p>

<pre><code>train_op = ...
for X, y in iterate_batches(X_train, y_train, you_batch_size, your_num_epochs):
    sess.run([train_op], feed_dict={X_tensor: X, y_tensor: y}
</code></pre>

<p>Here, <code>X_tensor</code> and <code>y_tensor</code> are <code>tf.placeholder</code>s for the data, that you have to specify in your network architecture.</p>

python tensorflow conv-neural-network