Find all-zero columns in pandas sparse matrix
<p>For example I have a coo_matrix A :</p>

<pre><code>import scipy.sparse as sp
A = sp.coo_matrix([3,0,3,0],
                  [0,0,2,0],
                  [2,5,1,0],
                  [0,0,0,0])
</code></pre>

<p>How can I get result [0,0,0,1], which indicates that first 3 columns contain non-zero values, only the 4th column is all zeros.</p>

<p>PS : cannot convert A to other type.<br>
PS2 : I tried using <code>np.nonzeros</code> but it seems that my implementation is not very elegant.</p>

<p><strong>Approach #1</strong> We could do something like this -</p>

<pre><code># Get the columns indices of the input sparse matrix
C = sp.find(A)[1]

# Use np.in1d to create a mask of non-zero columns. 
# So, we invert it and convert to int dtype for desired output.
out = (~np.in1d(np.arange(A.shape[1]),C)).astype(int)
</code></pre>

<p>Alternatively, to make the code shorter, we can use subtraction -</p>

<pre><code>out = 1-np.in1d(np.arange(A.shape[1]),C)
</code></pre>

<p>Step-by-step run -</p>

<p>1) Input array and sparse matrix from it :</p>

<pre><code>In [137]: arr             # Regular dense array
Out[137]: 
array([[3, 0, 3, 0],
       [0, 0, 2, 0],
       [2, 5, 1, 0],
       [0, 0, 0, 0]])

In [138]: A = sp.coo_matrix(arr) # Convert to sparse matrix as input here on
</code></pre>

<p>2) Get non-zero column indices :</p>

<pre><code>In [139]: C = sp.find(A)[1]

In [140]: C
Out[140]: array([0, 2, 2, 0, 1, 2], dtype=int32)
</code></pre>

<p>3) Use <code>np.in1d</code> to get mask of non-zero columns :</p>

<pre><code>In [141]: np.in1d(np.arange(A.shape[1]),C)
Out[141]: array([ True,  True,  True, False], dtype=bool)
</code></pre>

<p>4) Invert it :</p>

<pre><code>In [142]: ~np.in1d(np.arange(A.shape[1]),C)
Out[142]: array([False, False, False,  True], dtype=bool)
</code></pre>

<p>5) Finally convert to int dtype :</p>

<pre><code>In [143]: (~np.in1d(np.arange(A.shape[1]),C)).astype(int)
Out[143]: array([0, 0, 0, 1])
</code></pre>

<p>Alternative subtraction approach :</p>

<pre><code>In [145]: 1-np.in1d(np.arange(A.shape[1]),C)
Out[145]: array([0, 0, 0, 1])
</code></pre>

<p><strong>Approach #2</strong> Here's another way and possibly a faster one using <code>matrix-multiplication</code> -</p>

<pre><code>out = 1-np.ones(A.shape[0],dtype=bool)*A.astype(bool)
</code></pre>

<hr>

<p><strong>Runtime test</strong> </p>

<p>Let's test out all the posted approaches on a big and really sparse matrix -</p>

<pre><code>In [29]: A = sp.coo_matrix((np.random.rand(4000,4000)&gt;0.998).astype(int))

In [30]: %timeit 1-np.in1d(np.arange(A.shape[1]),sp.find(A)[1])
100 loops, best of 3: 4.12 ms per loop # Approach1

In [31]: %timeit 1-np.ones(A.shape[0],dtype=bool)*A.astype(bool)
1000 loops, best of 3: 771 Âµs per loop # Approach2

In [32]: %timeit 1 - (A.col==np.arange(A.shape[1])[:,None]).any(axis=1)
1 loops, best of 3: 236 ms per loop # @hpaulj's soln

In [33]: %timeit (A!=0).sum(axis=0)==0
1000 loops, best of 3: 1.03 ms per loop  # @jez's soln

In [34]: %timeit (np.sum(np.absolute(A.toarray()), 0) == 0) * 1
10 loops, best of 3: 86.4 ms per loop  # @wwii's soln 
</code></pre>

python numpy scipy sparse-matrix