Find all-zero columns in pandas sparse matrix
<p>For example I have a coo_matrix A :</p>

<pre><code>import scipy.sparse as sp
A = sp.coo_matrix([3,0,3,0],
                  [0,0,2,0],
                  [2,5,1,0],
                  [0,0,0,0])
</code></pre>

<p>How can I get result [0,0,0,1], which indicates that first 3 columns contain non-zero values, only the 4th column is all zeros.</p>

<p>PS : cannot convert A to other type.<br>
PS2 : I tried using <code>np.nonzeros</code> but it seems that my implementation is not very elegant.</p>

<p>Recent</p>

<p><a href="http://stackoverflow.com/questions/39683931/scipy-sparse-coo-matrix-how-to-fast-find-all-zeros-column-fill-with-1-and-norma">scipy.sparse.coo_matrix how to fast find all zeros column, fill with 1 and normalize</a></p>

<p>similar, except it wants to fill those columns with 1s and normalize them.</p>

<p>I immediately suggested the <code>lil</code> format of the transpose.  All-0 columns will be empty lists in this format.  But sticking with the <code>coo</code> format I suggested</p>

<pre><code>np.nonzero(~(Mo.col==np.arange(Mo.shape[1])[:,None]).any(axis=1))[0]
</code></pre>

<p>or for this 1/0 format</p>

<pre><code>1 - (Mo.col==np.arange(Mo.shape[1])[:,None]).any(axis=1)
</code></pre>

<p>which is functionally the same as:</p>

<pre><code>1 - np.in1d(np.arange(Mo.shape[1]),Mo.col)
</code></pre>

<p><code>sparse.find</code> converts the matrix to <code>csr</code> to sum duplicates and eliminate duplicates, and then back to <code>coo</code> to get the <code>data</code>, <code>row</code>, and <code>col</code> attributes (which it returns).</p>

<p><code>Mo.nonzero</code> uses <code>A.data != 0</code> to eliminate 0s before returning the <code>col</code> and <code>row</code> attributes.</p>

<p>The <code>np.ones(A.shape[0],dtype=bool)*A.astype(bool)</code> solution requires converting <code>A</code> to <code>csr</code> format for multiplication.  </p>

<p><code>(A!=0).sum(axis=0)</code> also converts to <code>csr</code> because column (or row) sum is done with a matrix multiplication.</p>

<p>So the no-convert requirement is unrealistic, at least within the bounds of sparse formats.  </p>

<p>===============</p>

<p>For Divakar's test case my <code>==</code> version is quite slow; it's ok with small ones, but creates too large of test array with the 1000 columns.</p>

<p>Testing on a matrix that is sparse enough to have a number of 0 columns:</p>

<pre><code>In [183]: Arr=sparse.random(1000,1000,.001)
In [184]: (1-np.in1d(np.arange(Arr.shape[1]),Arr.col)).any()
Out[184]: True
In [185]: (1-np.in1d(np.arange(Arr.shape[1]),Arr.col)).sum()
Out[185]: 367

In [186]: timeit 1-np.ones(Arr.shape[0],dtype=bool)*Arr.astype(bool)
1000 loops, best of 3: 334 Âµs per loop
In [187]: timeit 1-np.in1d(np.arange(Arr.shape[1]),Arr.col)
1000 loops, best of 3: 323 Âµs per loop
In [188]: timeit 1-(Arr.col==np.arange(Arr.shape[1])[:,None]).any(axis=1)
100 loops, best of 3: 3.9 ms per loop
In [189]: timeit (Arr!=0).sum(axis=0)==0
1000 loops, best of 3: 820 Âµs per loop
</code></pre>

python numpy scipy sparse-matrix