python filter 2d array by a chunk of data
<pre><code>import numpy as np

data = np.array([
    [20,  0,  5,  1],
    [20,  0,  5,  1],
    [20,  0,  5,  0],
    [20,  1,  5,  0],
    [20,  1,  5,  0],
    [20,  2,  5,  1],
    [20,  3,  5,  0],
    [20,  3,  5,  0],
    [20,  3,  5,  1],
    [20,  4,  5,  0],
    [20,  4,  5,  0],
    [20,  4,  5,  0]
])
</code></pre>

<p>I have the following 2d array. lets called the fields <code>a, b, c, d</code> in the above order where column <code>b</code> is like <code>id</code>. I wish to delete all cells that doesnt have atlist 1 appearance of the number "1" in column <code>d</code> for all cells with the same number in column <code>b</code> (same id) so after filtering i will have the following results:</p>

<pre><code>[[20  0  5  1]
 [20  0  5  1]
 [20  0  5  0]
 [20  2  5  1]
 [20  3  5  0]
 [20  3  5  0]
 [20  3  5  1]]
</code></pre>

<p>all rows with <code>b = 1</code> and <code>b = 4</code> have been deleted from the data</p>

<p>to sum up because I see answers that doesnt fit. we look at chunks of data by the <code>b</code> column. if a complete chunk of data doesnt have even one appearance of the number "1" in column <code>d</code> we delete all the rows of that <code>b</code> item. in the following example we can see a chunk of data with <code>b = 1</code> and <code>b = 4</code> ("id" = 1 and "id" = 4) that have 0 appearances of the number "1" in column <code>d</code>. thats why it gets deleted from the data </p>

<p><strong>Generic approach :</strong> Here's an approach using <a href="https://numeric.scipy.org/doc/numpy/reference/generated/numpy.unique.html" rel="nofollow"><code>np.unique</code></a> and <a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.bincount.html" rel="nofollow"><code>np.bincount</code></a> to solve for a generic case -</p>

<pre><code>unq,tags = np.unique(data[:,1],return_inverse=1)
goodIDs = np.flatnonzero(np.bincount(tags,data[:,3]==1)&gt;=1)
out = data[np.in1d(tags,goodIDs)]
</code></pre>

<p>Sample run -</p>

<pre><code>In [15]: data
Out[15]: 
array([[20, 10,  5,  1],
       [20, 73,  5,  0],
       [20, 73,  5,  1],
       [20, 31,  5,  0],
       [20, 10,  5,  1],
       [20, 10,  5,  0],
       [20, 42,  5,  1],
       [20, 54,  5,  0],
       [20, 73,  5,  0],
       [20, 54,  5,  0],
       [20, 54,  5,  0],
       [20, 31,  5,  0]])

In [16]: out
Out[16]: 
array([[20, 10,  5,  1],
       [20, 73,  5,  0],
       [20, 73,  5,  1],
       [20, 10,  5,  1],
       [20, 10,  5,  0],
       [20, 42,  5,  1],
       [20, 73,  5,  0]])
</code></pre>

<p><strong>Specific case approach :</strong>  If the second column data is always sorted and have sequential numbers starting from <code>0</code>, we can use a simplified version, like so -</p>

<pre><code>goodIDs = np.flatnonzero(np.bincount(data[:,1],data[:,3]==1)&gt;=1)
out = data[np.in1d(data[:,1],goodIDs)]
</code></pre>

<p>Sample run -</p>

<pre><code>In [44]: data
Out[44]: 
array([[20,  0,  5,  1],
       [20,  0,  5,  1],
       [20,  0,  5,  0],
       [20,  1,  5,  0],
       [20,  1,  5,  0],
       [20,  2,  5,  1],
       [20,  3,  5,  0],
       [20,  3,  5,  0],
       [20,  3,  5,  1],
       [20,  4,  5,  0],
       [20,  4,  5,  0],
       [20,  4,  5,  0]])

In [45]: out
Out[45]: 
array([[20,  0,  5,  1],
       [20,  0,  5,  1],
       [20,  0,  5,  0],
       [20,  2,  5,  1],
       [20,  3,  5,  0],
       [20,  3,  5,  0],
       [20,  3,  5,  1]])
</code></pre>

<p>Also, if <code>data[:,3]</code> always have ones and zeros, we can just use <code>data[:,3]</code> in place of <code>data[:,3]==1</code> in the above listed codes.</p>

<hr>

<p><strong>Benchmarking</strong> </p>

<p>Let's benchmark the vectorized approaches on the specific case for a larger array -</p>

<pre><code>In [69]: def logical_or_based(data): #@ Eric's soln
    ...:     b_vals = data[:,1]
    ...:     d_vals = data[:,3]
    ...:     is_ok = np.zeros(np.max(b_vals) + 1, dtype=np.bool_)
    ...:     np.logical_or.at(is_ok, b_vals, d_vals)
    ...:     return is_ok[b_vals]
    ...: 
    ...: def in1d_based(data):
    ...:     goodIDs = np.flatnonzero(np.bincount(data[:,1],data[:,3])!=0)
    ...:     out = np.in1d(data[:,1],goodIDs)
    ...:     return out
    ...: 

In [70]: # Setup input
    ...: data = np.random.randint(0,100,(10000,4))
    ...: data[:,1] = np.sort(np.random.randint(0,100,(10000)))
    ...: data[:,3] = np.random.randint(0,2,(10000))
    ...: 

In [71]: %timeit logical_or_based(data) #@ Eric's soln
1000 loops, best of 3: 1.44 ms per loop

In [72]: %timeit in1d_based(data)
1000 loops, best of 3: 528 Âµs per loop
</code></pre>

python arrays numpy