Trying to Remove for-Loops from Python code, Performing Operations with a Look-up Table On Matrices
<p>I feel like this is a similar problem to the one I asked before, but I can't figure it out. How can I convert these two lines of code into one line with no for-loop?</p>

<pre><code>for i in xrange(X.shape[0]):
  dW[:,y[i]] -= X[i]
</code></pre>

<p>In English, every row in matrix X should be subtracted from a corresponding column in matrix dW given by the vector y.</p>

<p>I should mention dW is DXC and X is NXD, so the transpose of X does not have the same shape as W, otherwise I could re-order the the rows of X, and take the transpose directly. However, it is possible for the columns in dW to have multiple corresponding rows which need to be subtracted.</p>

<p>I feel like I do not have a firm grasp of how indexing in python is supposed to work, which makes it difficult to remove unnecessary for-loops, or even to know what for-loops are possible to remove. </p>

<p>The straightforward way to vectorize would be:</p>

<pre><code>dW[:,y] -= X.T
</code></pre>

<p>Except, though not very obvious or well-documented, this will give problems with repeated indices in <code>y</code>. For these situations there is the <code>ufunc.at</code> method (elementwise operations in numpy are implemented as "ufunc's" or "universal functions"). Quote from <a href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.at.html" rel="nofollow">the docs</a>:</p>

<blockquote>
  <p>ufunc.at(a, indices, b=None)</p>
  
  <p>Performs unbuffered in place operation on operand âaâ for elements specified by âindicesâ. For addition ufunc, this method is equivalent to a[indices] += b, except that results are accumulated for elements that are indexed more than once. For example, a[[0,0]] += 1 will only increment the first element once because of buffering, whereas add.at(a, [0,0], 1) will increment the first element twice.</p>
</blockquote>

<p>So in your case:</p>

<pre><code>np.subtract.at(dW.T, y, X)
</code></pre>

<p>Unfortunately, <code>ufunc.at</code> is relatively inefficient as far as vectorization techniques go, so the speedup compared to the loop might not be that impressive.</p>

python numpy matrix indexing vectorization