Implement Bhattacharyya loss function using python layer Caffe
<p>Trying to implement my custom loss layer using python layer,caffe. I've used this <a href="https://github.com/BVLC/caffe/blob/master/examples/pycaffe/layers/pyloss.py" rel="nofollow">example</a> as the guide and have wrote the <code>forward</code> function as follow:</p>

<pre><code>    def forward(self,bottom,top):
        score = 0;
        self.mult[...] = np.multiply(bottom[0].data,bottom[1].data)
        self.multAndsqrt[...] = np.sqrt(self.mult)
        top[0].data[...] = -math.log(np.sum(self.multAndsqrt))
</code></pre>

<p>However, the second task, that is implementing the <code>backward</code> function is kinda much difficult for me as I'm totally unfamiliar with python. So please help me with coding the backward section. 
Here is the cost function and its derivative for stocashtic gradient decent to be implemented:</p>

<p><a href="https://i.stack.imgur.com/GXgGA.png" rel="nofollow"><img src="https://i.stack.imgur.com/GXgGA.png" alt="enter image description here"></a>
Thanks in advance.</p>

<p>Note that p[i] in the table indicates the <em>ith</em> output neuron value. </p>

<p>Lets say bottom[0].data is p, bottom[1].data is q and Db(p,q) denotes the Bhattacharyya Distance between p and q.</p>

<p>The only thing you need to do in your backward function is to compute the partial derivatives of Db with respect to its inputs (p and q) and store them in the respective bottom diff blobs:</p>

<p><a href="https://i.stack.imgur.com/Q3ckj.gif" rel="nofollow">diff_p = dDb(p,q)/dp</a><br>
<a href="https://i.stack.imgur.com/6s5Jl.gif" rel="nofollow">diff_q = dDb(p,q)/dq</a></p>

<p>so your backward function would look something like:</p>

<pre><code>def backward(self, top, propagate_down, bottom):
    if propagate_down[0]:
        bottom[0].diff[...] = # calculate dDb(p,q)/dp
    if propagate_down[1]:
        bottom[1].diff[...] = # calculate dDb(p,q)/dq
</code></pre>

<p>Note that you normally use the average (instad of the total) error of your batch. Then you would end up with something like this:</p>

<pre><code>def forward(self,bottom,top):
    self.mult[...] = np.multiply(bottom[0].data,bottom[1].data)
    self.multAndsqrt[...] = np.sqrt(self.mult)
    top[0].data[...] = -math.log(np.sum(self.multAndsqrt)) / bottom[0].num

def backward(self, top, propagate_down, bottom):
    if propagate_down[0]:
        bottom[0].diff[...] = # calculate dDb(p,q)/dp
                                / bottom[0].num
    if propagate_down[1]:
        bottom[1].diff[...] = # calculate dDb(p,q)/dq
                                / bottom[1].num
</code></pre>

<p>Once you calculated the partial derivatives of Db, you can insert them in the templates above as you did for the function of the forward pass.</p>

python python-2.7 caffe pycaffe