How does numpy broadcasting perform faster?
<p>In the following question,
<a href="http://stackoverflow.com/a/40056135/5714445">http://stackoverflow.com/a/40056135/5714445</a></p>

<p>Numpy's broadcasting provides a solution that's almost 6x faster than using np.setdiff1d() paired with np.view(). How does it manage to do this?</p>

<p>And using <code>A[~((A[:,None,:] == B).all(-1)).any(1)]</code> speeds it up even more. 
Interesting, but raises yet another question. How does this perform even better?</p>

<p>I would try to answer the second part of the question.</p>

<p>So, with it we are comparing :</p>

<pre><code>A[np.all(np.any((A-B[:, None]), axis=2), axis=0)]  (I)
</code></pre>

<p>and </p>

<pre><code>A[~((A[:,None,:] == B).all(-1)).any(1)]
</code></pre>

<p>To compare with a matching perspective against the first one, we could write down the second approach like this -</p>

<pre><code>A[(((~(A[:,None,:] == B)).any(2))).all(1)]         (II)
</code></pre>

<p>The major difference when considering performance, would be the fact that with the first one, we are getting non-matches with subtraction and then checking for non-zeros with <code>.any()</code>. Thus, <code>any()</code> is made to operate on an array of non-boolean dtype array. In the second approach, instead we are feeding it a boolean array obtained with <code>A[:,None,:] == B</code>.</p>

<p>Let's do a small runtime test to see how <code>.any()</code> performs on <code>int</code> dtype vs <code>boolean array</code> -</p>

<pre><code>In [141]: A = np.random.randint(0,9,(1000,1000)) # An int array

In [142]: %timeit A.any(0)
1000 loops, best of 3: 1.43 ms per loop

In [143]: A = np.random.randint(0,9,(1000,1000))&gt;5 # A boolean array

In [144]: %timeit A.any(0)
10000 loops, best of 3: 164 Âµs per loop
</code></pre>

<p>So, with close to <strong><code>9x</code></strong> speedup on this part, we see a huge advantage to use <code>any()</code> with boolean arrays. This I think was the biggest reason to make the second approach faster.</p>

python numpy