pandas.factorize on an entire data frame
<p><a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.factorize.html" rel="nofollow"><code>pandas.factorize</code></a> encodes input values as an enumerated type or categorical variable. </p>

<p>But how can I easily and efficiently convert many columns of a data frame? What about the reverse mapping step?</p>

<p>Example: This data frame contains columns with string values such as "type 2" which I would like to convert to numerical values - and possibly translate them back later.</p>

<p><a href="http://i.stack.imgur.com/MLATh.png" rel="nofollow"><img src="http://i.stack.imgur.com/MLATh.png" alt="enter image description here"></a></p>

<p>You can use <code>apply</code> if you need to <code>factorize</code> each column separately:</p>

<pre><code>df = pd.DataFrame({'A':['type1','type2','type2'],
                   'B':['type1','type2','type3'],
                   'C':['type1','type3','type3']})

print (df)
       A      B      C
0  type1  type1  type1
1  type2  type2  type3
2  type2  type3  type3

print (df.apply(lambda x: pd.factorize(x)[0]))
   A  B  C
0  0  0  0
1  1  1  1
2  1  2  1
</code></pre>

<p>If you need for the same string value the same numeric one:</p>

<pre><code>print (df.stack().rank(method='dense').unstack())
     A    B    C
0  1.0  1.0  1.0
1  2.0  2.0  3.0
2  2.0  3.0  3.0
</code></pre>

<hr>

<p>If you need to apply the function only for some columns, use a subset:</p>

<pre><code>df[['B','C']] = df[['B','C']].stack().rank(method='dense').unstack()
print (df)
       A    B    C
0  type1  1.0  1.0
1  type2  2.0  3.0
2  type2  3.0  3.0
</code></pre>

<p>Solution with <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.factorize.html" rel="nofollow"><code>factorize</code></a>:</p>

<pre><code>stacked = df[['B','C']].stack()
df[['B','C']] = pd.Series(stacked.factorize()[0], index=stacked.index).unstack()
print (df)
       A  B  C
0  type1  0  0
1  type2  1  2
2  type2  2  2
</code></pre>

<p>Translate them back is possible via <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.map.html" rel="nofollow"><code>map</code></a> by <code>dict</code>, where you need to remove duplicates by <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.drop_duplicates.html" rel="nofollow"><code>drop_duplicates</code></a>:</p>

<pre><code>vals = df.stack().drop_duplicates().values
b = [x for x in df.stack().drop_duplicates().rank(method='dense')]

d1 = dict(zip(b, vals))
print (d1)
{1.0: 'type1', 2.0: 'type2', 3.0: 'type3'}

df1 = df.stack().rank(method='dense').unstack()
print (df1)
     A    B    C
0  1.0  1.0  1.0
1  2.0  2.0  3.0
2  2.0  3.0  3.0

print (df1.stack().map(d1).unstack())
       A      B      C
0  type1  type1  type1
1  type2  type2  type3
2  type2  type3  type3
</code></pre>

python pandas dataframe machine-learning