Efficient way to read non-empty cells in a column in CSV file
<p>I have a large python file (>500,000 rows), and would like to read non-empty cells in a column in the dataframe (panda). Right now, I am doing this:</p>

<pre><code>df = pd.read_csv(filename)

myiter = []    

    for xiter, x in enumerate(df['Column_name']):

        if (np.isnan(x) == False):
            myiter.append(xiter)
</code></pre>

<p>Is there a more efficient way to do the same?</p>

<p>are they tagged as <code>NaN</code> in your <code>df</code>?</p>

<p>if yes then do</p>

<pre><code>df.dropna()
</code></pre>

python pandas dataframe large-files import-from-csv