How do "de-embed" words in TensorFlow
<p>I am trying to follow the tutorial for <a href="https://www.tensorflow.org/versions/r0.11/tutorials/recurrent/index.html#language-modeling" rel="nofollow">Language Modeling</a> on the TensorFlow site.  I see it runs and the cost goes down and it is working great, but I do not see any way to actually get the predictions from the model.  I tried following the instructions at <a href="http://stackoverflow.com/questions/37179218/getting-word-from-id-at-tensorflow-rnn-sample">this answer</a> but the tensors returned from session.run are floating point values like 0.017842259, and the dictionary maps words to integers so that does not work.</p>

<p>How can I get the predicted word from a tensorflow model? </p>

<p>Edit: I found this <a href="https://github.com/tensorflow/tensorflow/issues/97" rel="nofollow">explanation</a> after searching around, I just am not sure what x and y would be in the context of this example.  They don't seem to use the same conventions for this example as they do in the explanation.</p>

<p>The tensor you are mentioning is the <code>loss</code>, which defines how the network is training. For prediction, you need to access the tensor <code>probabilities</code> which contain the probabilities for the next word. If this was classification problem, you'd just do <code>argmax</code> to get the top probability. But, to also give lower probability words a chance of being generated,some kind of sampling is often used.</p>

<p>Edit: I assume the code you used is <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/models/rnn/ptb/ptb_word_lm.py" rel="nofollow">this</a>. In that case, if you look at line 148 (<code>logits</code>) which can be converted into probabilities by simply applying the <code>softmax</code> function to it -- like shown in the pseudocode in tensorflow website. Hope this helps.</p>

python tensorflow word2vec