Understanding Tensorflow LSTM Input shape
<p>I have a dataset X which consists <strong>N = 4000 samples</strong>, each sample consists of <strong>d = 2 features</strong> (continuous values) spanning back <strong>t = 10 time steps</strong>. I also have the  corresponding 'labels' of each sample which are also continuous values, at time step 11. </p>

<p>At the moment my dataset is in the shape X: [4000,20], Y: [4000].</p>

<p>I want to train an LSTM using TensorFlow to predict the value of Y (regression), given the 10 previous inputs of d features, but I am having a tough time implementing this in TensorFlow.</p>

<p>The main problem I have at the moment is understanding how TensorFlow is expecting the input to be formatted. I have seen various examples such as <a href="http://mourafiq.com/2016/05/15/predicting-sequences-using-rnn-in-tensorflow.html" rel="nofollow">this</a>, but these examples deal with one big string of continuous time series data. My data is different samples, each an independent time series.</p>

<p>Thank you.</p>

<p>The <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/nn.html#dynamic_rnn" rel="nofollow">documentation of <code>tf.nn.dynamic_rnn</code></a> states:</p>

<blockquote>
  <p><code>inputs</code>: The RNN inputs. If <code>time_major == False</code> (default), this must be a Tensor of shape: <code>[batch_size, max_time, ...]</code>, or a nested tuple of such elements.</p>
</blockquote>

<p>In your case, this means that the input should have a shape of <code>[batch_size, 10, 2]</code>. Instead of training on all 4000 sequences at once, you'd use only <code>batch_size</code> many of them in each training iteration. Something like the following should work (added reshape for clarity):</p>

<pre><code>batch_size = 32
# batch_size sequences of length 10 with 2 values for each timestep
input = get_batch(X, batch_size).reshape([batch_size, 10, 2])
# Create LSTM cell with state size 256. Could also use GRUCell, ...
# Note: state_is_tuple=False is deprecated;
# the option might be completely removed in the future
cell = tf.nn.rnn_cell.LSTMCell(256, state_is_tuple=True)
outputs, state = tf.nn.dynamic_rnn(cell,
                                   input,
                                   sequence_length=[10]*batch_size,
                                   dtype=tf.float32)
</code></pre>

<p>From the <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/nn.html#dynamic_rnn" rel="nofollow">documentation</a>, <code>outputs</code> will be of shape <code>[batch_size, 10, 256]</code>, i.e. one 256-output for each timestep. <code>state</code> will be a <a href="https://github.com/tensorflow/tensorflow/blob/r0.10/tensorflow/python/ops/rnn_cell.py#L238" rel="nofollow">tuple</a> of shapes <code>[batch_size, 256]</code>. You could predict your final value, one for each sequence, from that:</p>

<pre><code>predictions = tf.contrib.layers.fully_connected(state.h,
                                                num_outputs=1,
                                                activation_fn=None)
loss = get_loss(get_batch(Y).reshape([batch_size, 1]), predictions)
</code></pre>

<p>The number 256 in the shapes of <code>outputs</code> and <code>state</code> is determined by <code>cell.output_size</code> resp. <code>cell.state_size</code>. When creating the <code>LSTMCell</code> like above, these are the same. Also see the <a href="https://www.tensorflow.org/versions/r0.10/api_docs/python/rnn_cell.html#LSTMCell" rel="nofollow">LSTMCell documentation</a>.</p>

python tensorflow regression lstm