Numpy efficient indexing with varied size arrays
<p>Take a look at this piece of code:</p>

<pre><code>import numpy as np

a = np.random.random(10)

indicies = [
    np.array([1, 4, 3]),
    np.array([2, 5, 8, 7, 3]),
    np.array([1, 2]),
    np.array([3, 2, 1])
]

result = np.zeros(2)
result[0] = a[indicies[0]].sum()
result[1] = a[indicies[2]].sum()
</code></pre>

<p>Is there any way to get <code>result</code> more efficiently? In my case <code>a</code> is a very large array.</p>

<p>In other words I want to select elements from <code>a</code> with several varying size index arrays and then sum over them in one operation, resulting in a single array.</p>

<p>With your <code>a</code> and <code>indicies</code> list:</p>

<pre><code>In [280]: [a[i].sum() for i in indicies]
Out[280]: 
[1.3986792680307709,
 2.6354365193743732,
 0.83324677494990895,
 1.8195179021311731]
</code></pre>

<p>Which of course could wrapped in <code>np.array()</code>.</p>

<p>For a subset of the <code>indicies</code> items use:</p>

<pre><code>In [281]: [a[indicies[i]].sum() for i in [0,2]]
Out[281]: [1.3986792680307709, 0.83324677494990895]
</code></pre>

<p>A comment suggests <code>indicies</code> comes from an Adjacency matrix, possibly sparse.</p>

<p>I could recreate such an array with:</p>

<pre><code>In [289]: A=np.zeros((4,10),int)
In [290]: for i in range(4): A[i,indicies[i]]=1
In [291]: A
Out[291]: 
array([[0, 1, 0, 1, 1, 0, 0, 0, 0, 0],
       [0, 0, 1, 1, 0, 1, 0, 1, 1, 0],
       [0, 1, 1, 0, 0, 0, 0, 0, 0, 0],
       [0, 1, 1, 1, 0, 0, 0, 0, 0, 0]])
</code></pre>

<p>and use a matrix product (<code>np.dot</code>) to do the selection and sum:</p>

<pre><code>In [292]: A.dot(a)
Out[292]: array([ 1.39867927,  2.63543652,  0.83324677,  1.8195179 ])
</code></pre>

<p><code>A[[0,2],:].dot(a)</code> would use a subset of rows.</p>

<p>A sparse matrix version has that list of row indices:</p>

<pre><code>In [294]: Al=sparse.lil_matrix(A)
In [295]: Al.rows
Out[295]: array([[1, 3, 4], [2, 3, 5, 7, 8], [1, 2], [1, 2, 3]], dtype=object)
</code></pre>

<p>And a matrix product with that gives the same numbers:</p>

<pre><code>In [296]: Al*a
Out[296]: array([ 1.39867927,  2.63543652,  0.83324677,  1.8195179 ])
</code></pre>

python arrays numpy