TensorFlow simple operations: tensors vs Python variables
<p>I'm unsure about the practical differences between the 4 variations below (they all evaluate to the same value). My understanding is that if I call <code>tf</code>, it <em>will</em> create an operation on the graph, and otherwise it <em>might</em>. If I don't create the <code>tf.constant()</code> at the beginning, I believe that the constants will be created implicitly when doing the addition; but for <code>tf.add(a,b)</code> vs <code>a + b</code> where <code>a</code> and <code>b</code> are both Tensors (#1 and #3), I can see no difference besides the default naming (former is <code>Add</code> and the latter one is <code>add</code>). Can anyone shed some light on the differences between those, and when should one use each?</p>

<pre><code>## 1
a = tf.constant(1)
b = tf.constant(1)
x = tf.add(a, b)
with tf.Session() as sess:
    x.eval()

## 2
a = 1
b = 1
x = tf.add(a, b)
with tf.Session() as sess:
    x.eval()

## 3
a = tf.constant(1)
b = tf.constant(1)
x = a + b
with tf.Session() as sess:
    x.eval()

## 4
a = 1
b = tf.constant(1)
x = a + b
with tf.Session() as sess:
    x.eval()
</code></pre>

<p>The four examples you gave will all give the same result, and generate the same graph (if you ignore that some of the operation names in the graph are different). TensorFlow will convert many different Python objects into <code>tf.Tensor</code> objects when they are passed as arguments to TensorFlow operators, such as <code>tf.add()</code> here. The <code>+</code> operator is just a simple wrapper on <code>tf.add()</code>, and the overload is used when either the left-hand or right-hand argument is a <code>tf.Tensor</code> (or <code>tf.Variable</code>).</p>

<p>Given that you can just pass many Python objects to TensorFlow operators, why would you ever use <code>tf.constant()</code>? There are a few reasons:</p>

<ul>
<li><p>If you use the same Python object as the argument to multiple different operations, TensorFlow will convert it to a tensor multiple times, and represent each of those tensors in the graph. Therefore, if your Python object is a large NumPy array, you may run out of memory if you make too many copies of that array's data. In that case, you may wish to convert the array to a <code>tf.Tensor</code> once</p></li>
<li><p>Creating a <code>tf.constant()</code> explicitly allows you to set its <code>name</code> property, which can be useful for TensorBoard debugging and graph visualization. (Note though that the default TensorFlow ops will attempt to give a meaningful name to each automatically converted tensor, based on the name of the op's argument.)</p></li>
<li><p>Creating a <code>tf.constant()</code> explicitly allows you to set the exact element type of the tensor. TensorFlow will convert Python <code>int</code> objects to <code>tf.int32</code>, and <code>float</code> objects to <code>tf.float32</code>. If you want <code>tf.int64</code> or <code>tf.float64</code>, you can get this by passing the same value to <code>tf.constant()</code> and passing an explicit <code>dtype</code> argument.</p></li>
<li><p>The <code>tf.constant()</code> function also offers a useful feature when creating large tensors with a repeated value:</p>

<pre><code>c = tf.constant(17.0, shape=[1024, 1024], dtype=tf.float32)
</code></pre>

<p>The tensor <code>c</code> above represents 4 * 1024 * 1024 bytes of data, but TensorFlow will represent it compactly in the graph as a single float <code>17.0</code> plus shape information that indicates how it should be interpreted. If you have many large, filled constants in your graph, it can be more efficient to create them this way.</p></li>
</ul>

python tensorflow