pandas ordering columns misses values
<p>Not sure on the right title for this. But I have a need to take out a column from a dataframe, and show the top five results. The column is a mix of integers and n/a results. As an example I create a basic dataframe:</p>

<pre><code>regiona col1
a   n/a
a   1
a   200
b   208
b   400
b   560
b   600
c   800
c   1120
c   1200
c   1680
d   n/a
d   n/a
</code></pre>

<p>And so run:</p>

<pre><code>import pandas as pd
df = pd.read_csv('test_data.csv')
</code></pre>

<p>I then created a basic function so I could use this on different columns, so constructed:</p>

<pre><code>def max_search(indicator):
    displaced_count = df[df[indicator] != 'n/a']
    table = displaced_count.sort_values([indicator], ascending=[False])
    return table.head(5)
</code></pre>

<p>But when I run </p>

<pre><code>max_search('col1')
</code></pre>

<p>It returns:</p>

<pre><code>    regiona col1
7         c  800
6         b  600
5         b  560
4         b  400
3         b  208
</code></pre>

<p>So it misses anything greater than 800. The steps I think the function should be doing is:</p>

<ol>
<li>Filter out n/a valyes</li>
<li>Return the top five values. </li>
</ol>

<p>However, it is not returning anything over 800? Am I missing something very obvious?</p>

<p>Check your dataframe's <code>dtypes</code>, now it is <code>object</code>. So first make sure  <code>col1</code>'s datatype is numeric.
Use <code>na_values</code> at <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html" rel="nofollow"><code>pd.read_csv()</code></a> and your function will work as expected:</p>

<pre><code>df = pd.read_csv('test_data.csv', na_values='n/a')
# df.dtypes
</code></pre>

python pandas