pandas: slice dataframe based on NaN
<p>I have following dataframe <code>df</code></p>

<pre><code>prod_id prod_ref
10      ef3920
12      bovjhd
NaN     lkbljb
NaN     jknnkn
30      kbknkn
</code></pre>

<p>I am trying the following:</p>

<pre><code>df[df['prod_id'] != np.nan]
</code></pre>

<p>but I get exactly the same dataframe.</p>

<p>I would like to display</p>

<pre><code>prod_id prod_ref
10      ef3920
12      bovjhd
30      kbknkn
</code></pre>

<p>What am I doing wrong?</p>

<p>The problem is that <code>np.nan != np.nan</code> is <code>True</code> (alternatively, <code>np.nan == np.nan</code> is <code>False</code>). Pandas provides the <code>.dropna()</code> method to do what you want:</p>

<pre><code>df.dropna()
</code></pre>

<p>Output:</p>

<pre class="lang-none prettyprint-override"><code>   prod_id prod_ref
0     10.0   ef3920
1     12.0   bovjhd
4     30.0   kbknkn
</code></pre>

<p>By default, <code>.dropna()</code> will drop any row that has a <code>NaN</code> in any column. You can tweak this behavior in two ways:</p>

<ul>
<li>check only some columns using the <code>subset</code> argument, and</li>
<li>require that the row contains <code>NaN</code> in <em>all</em> columns (in the <code>subset</code>, if you are using it) using <code>how='all'</code> &mdash; the default is <code>how='any'</code>.</li>
</ul>

<p>You can check the <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.dropna.html" rel="nofollow">documentation</a>.</p>

python python-2.7 pandas