explode pandas dataframe by unique elements in columns (strings) and create contingency table?
<p>I have a pandas dataframe that I would like to do some analysis on, it looks like this:</p>

<pre><code>from pandas import DataFrame
a = DataFrame([{'var1': 'K802', 'var2': 'No Concatenation', 'var3':'73410'},
           {'var1': 'O342,O820,Z370', 'var2': '59514,01968', 'var3':'146010'},
          {'var1': 'Z094', 'var2': 'No Concatenation', 'var3':'233210'},
          {'var1': 'N920', 'var2': '58120', 'var3':'130910'},
          {'var1': 'S801,W2064,I219', 'var2': 'No Concatenation', 'var3':'93630'},
          {'var1': 'O987,O820,Z302,Z370', 'var2': '59514,01968,58611', 'var3':'146010'},
          {'var1': 'O987,O820,Z302,Z370,E115', 'var2': '59514,01968,58611', 'var3':'146020'},
          {'var1': 'N359,N319,J459', 'var2': '52281', 'var3':'113720'},
          {'var1': 'O342,O343,O820,Z370', 'var2': '59514,01968,59871', 'var3':'146010'},
          {'var1': 'J459,C449,E785,I10', 'var2': 'No Concatenation', 'var3':'43810'},
          {'var1': 'Z380,C780,C189,I270,J449,Z933', 'var2': 'No Concatenation', 'var3':'157520'}])
print a.var1
0                              K802
1                    O342,O820,Z370
2                              Z094
3                              N920
4                   S801,W2064,I219
5               O987,O820,Z302,Z370
6          O987,O820,Z302,Z370,E115
7                    N359,N319,J459
8               O342,O343,O820,Z370
9                J459,C449,E785,I10
10    Z380,C780,C189,I270,J449,Z933
Name: var1, dtype: object
</code></pre>

<p>It has been truncated as the csv file it came from has 1 million plus rows. The goal is to end up with something like this:</p>

<pre><code>b = DataFrame([{'K802':1, 'O342': 0, 'O820':0, 'Z370':0, 'Z094': 0, 'N920':0, 'S801':0, 'W2064': 0, 'I219':0},
           {'K802':0, 'O342': 1, 'O820':1, 'Z370':1, 'Z094': 0, 'N920':0, 'S801':0, 'W2064': 0, 'I219':0},
           {'K802':0, 'O342': 0, 'O820':0, 'Z370':0, 'Z094': 1, 'N920':0, 'S801':1, 'W2064': 0, 'I219':0},
           {'K802':0, 'O342': 0, 'O820':0, 'Z370':0, 'Z094': 0, 'N920':1, 'S801':0, 'W2064': 0, 'I219':0},
           {'K802':0, 'O342': 0, 'O820':0, 'Z370':0, 'Z094': 0, 'N920':0, 'S801':1, 'W2064': 1, 'I219':1}])
print b
   I219  K802  N920  O342  O820  S801  W2064  Z094  Z370
0     0     1     0     0     0     0      0     0     0
1     0     0     0     1     1     0      0     0     1
2     0     0     0     0     0     1      0     1     0
3     0     0     1     0     0     0      0     0     0
4     1     0     0     0     0     1      1     0     0
...
</code></pre>

<p>Basically, I would like to get a new column for each unique entry in the rows of <code>a.var1</code> then populate the columns with either a <code>1</code> for is present in that row or <code>0</code> for not present. I need to do this for <code>var1</code>, <code>var2</code>, and <code>var3</code> separately then join the three by the indices of the original <code>a</code> so that I can calculate frequencies and maybe some logistic regression.
I am new to pandas, and can not seem to figure out how to do this efficiently.</p>

<p>Any help would be appreciated.</p>

<p>You can use the <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.str.get_dummies.html" rel="nofollow"><code>get_dummies</code></a> method defined on pd.Series. It is more straightforward to use than <code>pd.get_dummies</code> function for this use case. You can then use pd.concat to combine the resulting dfs.</p>

<pre><code>pd.concat([a[col].str.get_dummies(',') for col in a], axis=1)
Out: 
    C189  C449  C780  E115  E785  I10  I219  I270  J449  J459  ...    \
0      0     0     0     0     0    0     0     0     0     0  ...     
1      0     0     0     0     0    0     0     0     0     0  ...     
2      0     0     0     0     0    0     0     0     0     0  ...     
3      0     0     0     0     0    0     0     0     0     0  ...     
4      0     0     0     0     0    0     1     0     0     0  ...     
5      0     0     0     0     0    0     0     0     0     0  ...     
6      0     0     0     1     0    0     0     0     0     0  ...     
7      0     0     0     0     0    0     0     0     0     1  ...     
8      0     0     0     0     0    0     0     0     0     0  ...     
9      0     1     0     0     1    1     0     0     0     1  ...     
10     1     0     1     0     0    0     0     1     1     0  ...     
</code></pre>

python pandas join split