Fill missing categorial values using pandas?
<p>I'd like to fill missing categorial cells with new values per column. For example:</p>

<pre><code>c1  c2  c3
a   nan  a
b   q    nan
c   d    nan
a   p    z
</code></pre>

<p>should become something like</p>

<pre><code>c1  c2  c3
a   n1   a
b   q    n2
c   d    n2
a   p    z
</code></pre>

<p>My current problem is that I am using DictVectorizer for categorials column, but it leaves NaNs as-is.</p>

<p>Fillna with some uniq string does what you want:</p>

<pre><code>categorial_data = pd.DataFrame({'sex': ['male', 'female', 'male', 'female'],
                                'nationality': ['American', 'European', float('nan'), 'European']})
print(categorial_data)
categorial_data=categorial_data.fillna('some_unique_string')
print('after replacement')
print(categorial_data)
encoder = DV(sparse = False)
encoded_data = encoder.fit_transform(categorial_data.T.to_dict().values())
print(encoded_data)
</code></pre>

<p>gives you</p>

<pre><code>  nationality     sex
0    American    male
1    European  female
2         NaN    male
3    European  female
after replacement
          nationality     sex
0            American    male
1            European  female
2  some_unique_string    male
3            European  female
[[ 1.  0.  0.  0.  1.]
 [ 0.  1.  0.  1.  0.]
 [ 0.  0.  1.  0.  1.]
 [ 0.  1.  0.  1.  0.]]
</code></pre>

python missing-data pandas