How to handle non-numeric entries in an integer valued column
<p>I have a dataframe <code>df</code> and one of the columns <code>count</code> is contains strings. These strings are mostly convertable to integers (e.g. <code>0006</code>) which is what I will do with them. However some of the entries in <code>count</code> are blank strings of spaces. How can I </p>

<ul>
<li>Drop all the rows where the <code>count</code> value is a blank string.</li>
<li>Substitute all the blank values in that column with some numeric value of my choice.</li>
</ul>

<p>The dataframe is very large if there are particularly efficient ways of doing this.</p>

<p>Use <code>dropna</code> or <code>fillna</code> after <code>pd.to_numeric(errosr='coerce')</code></p>

<p>consider a pandas series <code>s</code></p>

<pre><code>s = pd.Series(np.random.choice(('0001', ''), 1000000), name='intish')
</code></pre>

<p><strong><em>drop method 1</em></strong> (less robust)</p>

<pre><code>s[s != ''].astype(int)
</code></pre>

<p><strong><em>drop method 2</em></strong> (more robust)</p>

<pre><code>pd.to_numeric(s, 'coerce').dropna().astype(int)
</code></pre>

<p><strong><em>drop timing</em></strong></p>

<p><a href="http://i.stack.imgur.com/Tqwgp.png" rel="nofollow"><img src="http://i.stack.imgur.com/Tqwgp.png" alt="enter image description here"></a></p>

<p><em>More robust method is faster</em></p>

<hr>

<p><strong><em>fill method 1</em></strong></p>

<pre><code>pd.to_numeric(s, 'coerce').fillna(0).astype(int)
</code></pre>

<p><strong><em>fill method 2</em></strong></p>

<pre><code>s.where(s.astype(bool), 0).astype(int)
</code></pre>

<p><strong><em>fill timing</em></strong></p>

<p><a href="http://i.stack.imgur.com/jvMGG.png" rel="nofollow"><img src="http://i.stack.imgur.com/jvMGG.png" alt="enter image description here"></a></p>

<p><em>Takes same amount of time as dropping</em></p>

python pandas