How do "de-embed" words in TensorFlow
<p>I am trying to follow the tutorial for <a href="https://www.tensorflow.org/versions/r0.11/tutorials/recurrent/index.html#language-modeling" rel="nofollow">Language Modeling</a> on the TensorFlow site.  I see it runs and the cost goes down and it is working great, but I do not see any way to actually get the predictions from the model.  I tried following the instructions at <a href="http://stackoverflow.com/questions/37179218/getting-word-from-id-at-tensorflow-rnn-sample">this answer</a> but the tensors returned from session.run are floating point values like 0.017842259, and the dictionary maps words to integers so that does not work.</p>

<p>How can I get the predicted word from a tensorflow model? </p>

<p>Edit: I found this <a href="https://github.com/tensorflow/tensorflow/issues/97" rel="nofollow">explanation</a> after searching around, I just am not sure what x and y would be in the context of this example.  They don't seem to use the same conventions for this example as they do in the explanation.</p>

<p>So after going through a bunch of other similar posts I figured this out.  First, the code explained in the documentation is not the same as the code on the GitHub repository.  The current code works by initializing models with data inside instead of passing data to the model as it goes along.</p>

<p>So basically to accomplish what I was trying to do, I reverted my code to commit <a href="https://github.com/tensorflow/tensorflow/blob/9274f5aa478879386870bb0cb889e432d4aee330/tensorflow/models/rnn/ptb/ptb_word_lm.py" rel="nofollow">9274f5a</a> (also do the same for reader.py).  Then I followed the steps taken in <a href="http://stackoverflow.com/questions/36286594/predicting-the-next-word-using-the-lstm-ptb-model-tensorflow-example?rq=1">this post</a> to get the <code>probabilities</code> tensor in my <code>run_epoch</code> function.  Additionally, I followed <a href="http://stackoverflow.com/questions/37179218/getting-word-from-id-at-tensorflow-rnn-sample?noredirect=1&amp;lq=1">this answer</a> to pass the <code>vocabulary</code> to my <code>main</code> function.  From there, I inverted the dict using <code>vocabulary = {v: k for k, v in vocabulary.items()}</code> and passed it to <code>run_epoch</code>.</p>

<p>Finally, we can get the predicted word in <code>run_epoch</code> by running <code>current_word = vocabulary[np.argmax(prob, 1)]</code> where <code>prob</code> is the tensor returned from <code>session.run()</code></p>

<p>Edit:  Reverting the code as such should not be a permanent solution and I definitely recommend using @Prophecies answer above to get the <code>probabilities</code> tensor.  However, if you want to get the word mapping, you will need to pass the vocabulary as I did here.</p>

python tensorflow word2vec