Python Pandas: select column with the number of unique values greater than 10
<p>In R, we can use <code>sapply</code> to extract columns with the number of unique values greater than 10 by:</p>

<pre><code>X[,  sapply(X,  function(x) length(unique(x)))  &gt;=10]
</code></pre>

<p>How can we do the same thing in Python Pandas?</p>

<p>Also, how can we choose columns with missing proportion less than 10% like what we can do in R:</p>

<pre><code>X[,  sapply(X,  function(x) sum(is.na(x))/length(x)  )  &lt; 0.1]
</code></pre>

<p>Thanks.</p>

<p>You can use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.nunique.html" rel="nofollow"><code>nunique</code></a> with <code>apply</code>, because it works only with <code>Series</code>:</p>

<pre><code>print (df.ix[:, df.apply(lambda x: x.nunique()) &gt;= 10])
</code></pre>

<p>and second <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.isnull.html" rel="nofollow"><code>isnull</code></a> with <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.mean.html" rel="nofollow"><code>mean</code></a>:</p>

<pre><code>print (df.ix[:, df.isnull().mean() &lt;  0.1])
</code></pre>

<p>Sample:</p>

<pre><code>df = pd.DataFrame({'A':[1,np.nan,3],
                   'B':[4,4,np.nan],
                   'C':[7,8,9],
                   'D':[3,3,5]})

print (df)
     A    B  C  D
0  1.0  4.0  7  3
1  NaN  4.0  8  3
2  3.0  NaN  9  5
</code></pre>



<pre><code>print (df.ix[:, df.apply(lambda x: x.nunique()) &gt;= 2])
     A  C  D
0  1.0  7  3
1  NaN  8  3
2  3.0  9  5

print (df.isnull().sum())
A    1
B    1
C    0
D    0
dtype: int64

print (df.isnull().sum() / len(df.index))

A    0.333333
B    0.333333
C    0.000000
D    0.000000
dtype: float64

print (df.isnull().mean())
A    0.333333
B    0.333333
C    0.000000
D    0.000000
dtype: float64


print (df.ix[:, df.isnull().sum() / len(df.index) &lt;  0.1])
   C  D
0  7  3
1  8  3
2  9  5
</code></pre>

<p>Or:</p>

<pre><code>print (df.ix[:, df.isnull().mean() &lt;  0.1])
   C  D
0  7  3
1  8  3
2  9  5
</code></pre>

python pandas sapply