broadcasting a comparison of a column of 2d array with multiple columns
<p>What's the right numpy syntax to compare one column against others in a 2d ndarray?  </p>

<p>After reading <a href="http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html" rel="nofollow">some</a> <a href="http://scipy.github.io/old-wiki/pages/EricsBroadcastingDoc" rel="nofollow">docs</a> on array broadcasting, I am still not quite sure what the correct way to do this is.</p>

<p><strong>Example:</strong>  Suppose I have a 2d array of goals scored by each player (row) in each game (column).  </p>

<pre><code># goals = number of goals scored by ith player in jth game (NaN if player did not play)
                           # column = game
goals = np.array([ [np.nan, 0,      1],   # row = player
                   [     1, 2,      0],
                   [     0, 0, np.nan],
                   [np.nan, 1,      1],
                   [     0, 0,      1] ])
</code></pre>

<p>I want to know if, in the final game, the player achieved a personal record by scoring more goals than she did in any previous game, ignoring games in which she did not appear (represented as <code>nan</code>).  I expect <code>True</code> for only the first and last players in the array.  </p>

<p>Just writing <code>goals[:,2] &gt; goals[:,:2]</code> returns the <code>ValueError: operands could not be broadcast together with shapes (5,) (5,2)</code></p>

<p><strong>What I tried:</strong> I know that I can manually stretch the <code>(5,)</code> into <code>(5,2)</code> with <code>np.newaxis</code>.  So this works:</p>

<pre><code>with np.errstate(invalid='ignore'):
  personalBest= ( np.isnan(goals[:,:2]) | 
                  (goals[:,2][:,np.newaxis] &gt; goals[:,:2] ) 
                 ).all(axis=1)

print(personalBest) # returns desired solution
</code></pre>

<p>Is there a less hacky, more idiomatically numpy way to write this?</p>

<p>Just focusing on the <code>newaxis</code> bit:</p>

<pre><code>In [332]: goals = np.arange(12).reshape(3,4)
In [333]: goals[:,2]&gt;goals[:,:2]
...
ValueError: operands could not be broadcast together with shapes (3,) (3,2)
</code></pre>

<p>So the goal is to make the 1st array of shape (3,1) so it can be broadcast against the (3,2):</p>

<p>We can index with a list or slice:  <code>goals[:,2:3]</code> works as well</p>

<pre><code>In [334]: goals[:,[2]]&gt;goals[:,:2]
Out[334]: 
array([[ True,  True],
       [ True,  True],
       [ True,  True]], dtype=bool)
</code></pre>

<p>we can explicitly add the <code>newaxis</code> (common)</p>

<pre><code>In [335]: goals[:,2][:,None]&gt;goals[:,:2]
Out[335]: 
array([[ True,  True],
       [ True,  True],
       [ True,  True]], dtype=bool)
</code></pre>

<p>we can combine the two indexing operations (this isn't seen as frequently)</p>

<pre><code>In [336]: goals[:,2,None]&gt;goals[:,:2]
Out[336]: 
array([[ True,  True],
       [ True,  True],
       [ True,  True]], dtype=bool)
</code></pre>

<p>we can explicitly reshape:</p>

<pre><code>In [339]: goals[:,2].reshape(-1,1)&gt;goals[:,:2]
Out[339]: 
array([[ True,  True],
       [ True,  True],
       [ True,  True]], dtype=bool)
</code></pre>

<p>I don't think the execution times differ significantly.  These are all good <code>numpy</code> code.</p>

<p>========</p>

<p>If the 2 arrays were (3,) and (2,3), we wouldn't need any of this.  The numpy broadcasting automatically expands the first to (1,3).  In effect <code>x[None,:]</code> is automatic, but <code>x[:,None]</code> is not.</p>

python arrays numpy numpy-broadcasting