How to directly set the gradient of a layer before backpropagation?
<p>Imagine a tiny network defined as follows, where linear is a typical helper function defining TensorFlow variables for a weight matrix and activation function:</p>

<p><code>final_layer = linear(linear(_input,10,tf.nn.tanh),20)</code></p>

<p>Normally this would be optimized via gradient descent on a loss:</p>

<p><code>loss = tf.reduce_sum(tf.square(final_layer - _target))
train_step = tf.train.AdamOptimizer().minimmize(loss)</code></p>

<p>But assume I'm getting the derivatives of the loss w.r.t. final_layer from an external  source (e.g. a tf.placeholder named _deriv). How can I use this gradient information with one of the builtin optimizers to backpropagate and update the network parameters?</p>

<p>The workaround I'm currently using is to construct an artificial loss consisting of the inner product between _deriv and final_layer (since the derivatives of this loss w.r.t. final_layer will be equal to _deriv). </p>

<p><code>loss = tf.reduce_sum(final_layer*_deriv)
train_step = tf.train.AdamOptimizer().minimmize(loss)</code></p>

<p>This is very wasteful though, as it needs to do this unnecessary inner product and calculate its derivative for every training step even though I already know this information. Is there a better way?</p>

<p>For those thinking this an odd thing to need to do, it is necessary for implementing <a href="https://arxiv.org/abs/1608.05343" rel="nofollow">synthetic gradients</a>.</p>

<p><code>tf.gradients</code> provides this functionality via its <code>grad_ys</code> argument, see <a href="https://www.tensorflow.org/versions/r0.11/api_docs/python/train.html#gradients" rel="nofollow">here</a>. In your case, <code>tf.gradients([final_layer], list_of_variables, grad_ys=[_deriv])</code> would compute the gradients you want.</p>

<p>Unfortunately, it looks like the build-in optimizers don't pass a <code>grad_ys</code> argument to <code>tf.gradients</code>. You might have to hack something into the <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/training/optimizer.py#L200" rel="nofollow">compute_gradients</a> method of the optimizer class.</p>

python tensorflow backpropagation gradient-descent