How can I use a 2D array of boolean rows to filter another 2D array?
<p>I have some data in a (3, m) array.</p>

<p>I have another array of masks in (n, 3) shape.  The rows of this mask are boolean filters that need to be applied on the data array before performing some function.  Is there a vectorized way to apply the filter and compute the function?</p>

<p>Here's an example using a loop for clarity, assuming the function is a mean().  I'd like to do this using purely Numpy (without the list comprehension).</p>

<p>(Obviously, the sizes of the arrays are much larger in reality.)</p>

<pre><code>import numpy as np

data = np.array([
       [ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11]
    ])

masks = np.array([
        [True, True, False],
        [False, True, False],
        [False, True, True],
        [True, False, False],
        [True, False, True]
    ])

means = np.array([data[mask].mean(axis=0) for mask in masks])

# means
array([[ 2.,  3.,  4.,  5.],
       [ 4.,  5.,  6.,  7.],
       [ 6.,  7.,  8.,  9.],
       [ 0.,  1.,  2.,  3.],
       [ 4.,  5.,  6.,  7.]])
</code></pre>

<p>So, after playing with it for a while, seems this kind of broadcasting works for mean() as the function specifically:</p>

<pre><code>means = (masks[:, :, np.newaxis] * data).sum(axis=1) / masks.sum(axis=1)[:, np.newaxis]

# means
array([[ 2.,  3.,  4.,  5.],
       [ 4.,  5.,  6.,  7.],
       [ 6.,  7.,  8.,  9.],
       [ 0.,  1.,  2.,  3.],
       [ 4.,  5.,  6.,  7.]])
</code></pre>

<p>And for other functions more generally, you could use this format (where mean() can be replaced with desired function):</p>

<pre><code>means = (masks[:, :, np.newaxis] * data).mean(axis=1) * masks.shape[1] / masks.sum(axis=1)[:, np.newaxis]

# means
array([[ 2.,  3.,  4.,  5.],
       [ 4.,  5.,  6.,  7.],
       [ 6.,  7.,  8.,  9.],
       [ 0.,  1.,  2.,  3.],
       [ 4.,  5.,  6.,  7.]])
</code></pre>

python performance numpy vectorization