How do I deal with Pandas Series data type that has NaN?
<p>What happens when using max() and min() on pandas.core.series.Series type that has NaN in it? Is this a bug? See below,</p>

<hr>

<pre><code>%matplotlib inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

mydata = pd.DataFrame(np.random.standard_normal((100,1)), columns=['No NaN'])
mydata['Has NaN'] = mydata['No NaN'] / mydata['No NaN'].shift(1)

# Both return NaN!
print(min(mydata['Has NaN']), max(mydata['Has NaN']))
# Still why False? Isn't float('nan') a singleton like None?
print(min(mydata['Has NaN']) == max(mydata['Has NaN']))
# But this time works well!
print(min([1, 2, 3, float('nan')]))

print('\n')

# When Series data type that has NaN bumps into min() and max(), what should 
#  I do? E.g.,
try: 
    n, bins, patches = plt.hist(mydata['Has NaN'], 10)
except ValueError as e:
    print(e, '\nSeems "range" argument in hist() has problem!')
</code></pre>

<p>First, you shouldn't use the Python built-in <code>max</code> or <code>min</code> when dealing with <code>pandas</code> or <code>numpy</code>, especially when you are working with <code>nan</code>.</p>

<p>Since 'nan' is the first item of <code>mydata['Has NaN']</code>, it is never replaced in either <code>max</code> or <code>min</code> because (as stated in the <a href="https://docs.python.org/3/reference/expressions.html#value-comparisons" rel="nofollow">docs</a>):</p>

<blockquote>
  <p>The not-a-number values float('NaN') and Decimal('NaN') are special.
  They are identical to themselves (x is x is true) but are not equal to
  themselves (x == x is false). Additionally, comparing any number to a
  not-a-number value will return False. For example, both 3 &lt;
  float('NaN') and float('NaN') &lt; 3 will return False.</p>
</blockquote>

<p>Instead, use the <code>pandas</code> <code>max</code> and <code>min</code> methods:</p>

<pre><code>In [4]: mydata['Has NaN'].min()
Out[4]: -176.9844930355774

In [5]: mydata['Has NaN'].max()
Out[5]: 12.684033138603787
</code></pre>

<p>With regards to the histogram, it seems this is a known issue with <code>plt.hist</code>, see <a href="https://github.com/matplotlib/matplotlib/issues/6483" rel="nofollow">here</a> and <a href="https://github.com/matplotlib/matplotlib/issues/6992" rel="nofollow">here</a>.</p>

<p>It should be fairly straightforward to deal with for now, though:</p>

<pre><code>n, bins, patches = plt.hist(mydata['Has NaN'][~mydata['Has NaN'].isnull()], 10)
</code></pre>

<p><a href="http://i.stack.imgur.com/ytxEV.png" rel="nofollow"><img src="http://i.stack.imgur.com/ytxEV.png" alt="enter image description here"></a></p>

python pandas matplotlib dataframe