broadcasting a comparison of a column of 2d array with multiple columns
<p>What's the right numpy syntax to compare one column against others in a 2d ndarray?  </p>

<p>After reading <a href="http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html" rel="nofollow">some</a> <a href="http://scipy.github.io/old-wiki/pages/EricsBroadcastingDoc" rel="nofollow">docs</a> on array broadcasting, I am still not quite sure what the correct way to do this is.</p>

<p><strong>Example:</strong>  Suppose I have a 2d array of goals scored by each player (row) in each game (column).  </p>

<pre><code># goals = number of goals scored by ith player in jth game (NaN if player did not play)
                           # column = game
goals = np.array([ [np.nan, 0,      1],   # row = player
                   [     1, 2,      0],
                   [     0, 0, np.nan],
                   [np.nan, 1,      1],
                   [     0, 0,      1] ])
</code></pre>

<p>I want to know if, in the final game, the player achieved a personal record by scoring more goals than she did in any previous game, ignoring games in which she did not appear (represented as <code>nan</code>).  I expect <code>True</code> for only the first and last players in the array.  </p>

<p>Just writing <code>goals[:,2] &gt; goals[:,:2]</code> returns the <code>ValueError: operands could not be broadcast together with shapes (5,) (5,2)</code></p>

<p><strong>What I tried:</strong> I know that I can manually stretch the <code>(5,)</code> into <code>(5,2)</code> with <code>np.newaxis</code>.  So this works:</p>

<pre><code>with np.errstate(invalid='ignore'):
  personalBest= ( np.isnan(goals[:,:2]) | 
                  (goals[:,2][:,np.newaxis] &gt; goals[:,:2] ) 
                 ).all(axis=1)

print(personalBest) # returns desired solution
</code></pre>

<p>Is there a less hacky, more idiomatically numpy way to write this?</p>

<p>You could do something like this -</p>

<pre><code>np.flatnonzero((goals[:,None,-1] &gt; goals[:,:-1]).any(1))
</code></pre>

<p>Let's go through it in steps.</p>

<p><strong>Step #1:</strong> We are introducing a new axis on the last-column sliced version to keep it as <code>2D</code> with the last axis being a singleton dimension/axis. The idea is to compare each of its element against all elements in that row except the element itself :</p>

<pre><code>In [3]: goals[:,None,-1]
Out[3]: 
array([[  1.],
       [  0.],
       [ nan],
       [  1.],
       [  1.]])

In [4]: goals[:,None,-1].shape # Check the shapes for broadcasting alignment
Out[4]: (5, 1)

In [5]: goals.shape
Out[5]: (5, 3)
</code></pre>

<p><strong>Step #2:</strong> Next up, we are actually performing the comparison against all the columns of the array skipping the last column itself as that's part of the sliced version obtained earlier -</p>

<pre><code>In [7]: goals[:,None,-1] &gt; goals[:,:-1]
Out[7]: 
array([[False,  True],
       [False, False],
       [False, False],
       [False, False],
       [ True,  True]], dtype=bool)
</code></pre>

<p><strong>Step #3:</strong> Then, we are checking if there's ANY match along each row -</p>

<pre><code>In [8]: (goals[:,None,-1] &gt; goals[:,:-1]).any(axis=1)
Out[8]: array([ True, False, False, False,  True], dtype=bool)
</code></pre>

<p><strong>Step #4:</strong> Finally, getting the matching indices with <code>np.flatnonzero</code> -</p>

<pre><code>In [9]: np.flatnonzero((goals[:,None,-1] &gt; goals[:,:-1]).any(axis=1))
Out[9]: array([0, 4])
</code></pre>

python arrays numpy numpy-broadcasting