TensorFlow, Python, Sharing Variables, initialize at top
<p>I'm having an issue in TensorFlow with Sharing Variables with the Python API.</p>

<p>I've read the official documentation (<a href="https://www.tensorflow.org/versions/r0.10/how_tos/variable_scope/index.html" rel="nofollow">https://www.tensorflow.org/versions/r0.10/how_tos/variable_scope/index.html</a>), but I still can't figure out what is going on.</p>

<p>I've written below a minimal working example to illustrate the issue.</p>

<p>In a nutshell, I'd like the code below do to the following:</p>

<p>1) Initialize one variable "fc1/w" immediately after I create the session,</p>

<p>2) Create a npy array "x_npy" to feed into a placeholder "x",</p>

<p>3) Run an operation "y", which should realize that the variable "fc1/w" is already created, and then use that variable values (rather than initialize new ones) to compute its output.</p>

<p>4) Please note that I added the flag ", reuse=True" in the variable scope in the function "linear", but that doesn't seem to help, since I keep getting the error:</p>

<pre><code>ValueError: Variable fc1/w does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?
</code></pre>

<p>This is quite confusing since if I were to remove the flag ", reuse=True", then TensorFlow would tell me that the variable does exist:</p>

<pre><code>ValueError: Variable fc1/w already exists, disallowed. Did you mean to set reuse=True in VarScope?
</code></pre>

<p>5) Please note that I'm working with a larger code base, and I'd really like to be able to use the Sharing Variables capability, rather than come up with a hack without using Sharing Variables that might solve the particular example code I wrote below, but might not generalize well.</p>

<p>6) Finally, please note also that I'd really like to keep separated the creation of the graph from it's evaluation. In particular, I would not like to use "tf.InteractiveSession()" or create "y" in the session scope, i.e., below: "with tf.Session() as sess:".</p>

<p>This is my first post on Stack Overflow, and I'm quite new to TensorFlow, so please accept my apologies if the question is not completely clear. In any case, I'd be happy to provide more details or clarify any aspect further.</p>

<p>Thank you in advance.</p>

<pre><code>import tensorflow as tf
import numpy as np


def linear(x_, output_size, non_linearity, name):
    with tf.variable_scope(name, reuse=True):
        input_size = x_.get_shape().as_list()[1]
        # If doesn't exist, initialize "name/w" randomly:
        w = tf.get_variable("w", [input_size, output_size], tf.float32,
                            tf.random_normal_initializer())
        z = tf.matmul(x_, w)
        return non_linearity(z)


def init_w(name, w_initializer):
    with tf.variable_scope(name):
        w = tf.get_variable("w", initializer=w_initializer)
        return tf.initialize_variables([w])


batch_size = 1
fc1_input_size = 7
fc1_output_size = 5

# Initialize with zeros
fc1_w_initializer = tf.zeros([fc1_input_size, fc1_output_size])

#
x = tf.placeholder(tf.float32, [None, fc1_input_size])

#
y = linear(x, fc1_output_size, tf.nn.softmax, "fc1")

with tf.Session() as sess:

    # Initialize "fc1/w" with zeros.
    sess.run(init_w("fc1", fc1_w_initializer))

    # Create npy array to feed into placeholder x
    x_npy = np.arange(batch_size * fc1_input_size, dtype=np.float32).reshape((batch_size, fc1_input_size))

    # Run y, and print result.
    print(sess.run(y, dict_feed={x: x_npy}))
</code></pre>

<p>It seems like the call to tf.variable_scope() finds the variable scope/w even if you run it in an empty session. I have cleaned up your code to demonstrate.</p>

<pre><code>import tensorflow as tf
import numpy as np


def shared_layer(x, output_size, non_linearity, name):
    with tf.variable_scope(name):
        input_size = x.get_shape().as_list()[1]
        # If doesn't exist, initialize "name/w" randomly:
        w = tf.get_variable("w", [input_size, output_size], tf.float32,
                            tf.random_normal_initializer())
        z = tf.matmul(x, w)
        return non_linearity(z)

def shared_init(sess, scope, var, initializer):
    with tf.variable_scope(scope, reuse=True):
        w = tf.get_variable(var, initializer=initializer)
        sess.run(tf.initialize_variables([w]))

layer_input_size = 2
layer_output_size = 2

w_init = tf.zeros([layer_input_size, layer_output_size])

x = tf.placeholder(tf.float32, [None, layer_input_size])
y = shared_layer(x, layer_output_size, tf.nn.softmax, "scope")

with tf.Session() as sess:
    shared_init(sess, "scope", "w", w_init)
    with tf.variable_scope("scope", reuse=True):
        print sess.run(tf.get_variable("w", [2,2]))
</code></pre>

python tensorflow