python filter 2d array by a chunk of data
<pre><code>import numpy as np

data = np.array([
    [20,  0,  5,  1],
    [20,  0,  5,  1],
    [20,  0,  5,  0],
    [20,  1,  5,  0],
    [20,  1,  5,  0],
    [20,  2,  5,  1],
    [20,  3,  5,  0],
    [20,  3,  5,  0],
    [20,  3,  5,  1],
    [20,  4,  5,  0],
    [20,  4,  5,  0],
    [20,  4,  5,  0]
])
</code></pre>

<p>I have the following 2d array. lets called the fields <code>a, b, c, d</code> in the above order where column <code>b</code> is like <code>id</code>. I wish to delete all cells that doesnt have atlist 1 appearance of the number "1" in column <code>d</code> for all cells with the same number in column <code>b</code> (same id) so after filtering i will have the following results:</p>

<pre><code>[[20  0  5  1]
 [20  0  5  1]
 [20  0  5  0]
 [20  2  5  1]
 [20  3  5  0]
 [20  3  5  0]
 [20  3  5  1]]
</code></pre>

<p>all rows with <code>b = 1</code> and <code>b = 4</code> have been deleted from the data</p>

<p>to sum up because I see answers that doesnt fit. we look at chunks of data by the <code>b</code> column. if a complete chunk of data doesnt have even one appearance of the number "1" in column <code>d</code> we delete all the rows of that <code>b</code> item. in the following example we can see a chunk of data with <code>b = 1</code> and <code>b = 4</code> ("id" = 1 and "id" = 4) that have 0 appearances of the number "1" in column <code>d</code>. thats why it gets deleted from the data </p>

<p>code: </p>

<pre><code>import numpy as np

my_list = [[20,0,5,1],
    [20,0,5,1],
    [20,0,5,0],
    [20,1,5,0],
    [20,1,5,0],
    [20,2,5,1],
    [20,3,5,0],
    [20,3,5,0],
    [20,3,5,1],
    [20,4,5,0],
    [20,4,5,0],
    [20,4,5,0]]

all_ids = np.array(my_list)[:,1]
unique_ids = np.unique(all_ids)
indices = [np.where(all_ids==ui)[0][0] for ui in unique_ids ]

final = []
for id in unique_ids:
    try:
        tmp_group = my_list[indices[id]:indices[id+1]]
    except:
        tmp_group = my_list[indices[id]:]
    if 1 in np.array(tmp_group)[:,3]:
        final.extend(tmp_group)

print np.array(final)
</code></pre>

<p>result: </p>

<pre><code>[[20  0  5  1]
 [20  0  5  1]
 [20  0  5  0]
 [20  2  5  1]
 [20  3  5  0]
 [20  3  5  0]
 [20  3  5  1]]
</code></pre>

python arrays numpy