Pandas Column differences, containing lists
<p>I have a data frame where the columns values are list and want to find the differences between two columns, or in other words I want to find all the elements in column A which is not there in column B.</p>

<pre><code>data={'NAME':['JOHN','MARY','CHARLIE'],
  'A':[[1,2,3],[2,3,4],[3,4,5]],
  'B':[[2,3,4],[3,4,5],[4,5,6]]}
df=pd.DataFrame(data)
df=df[['NAME','A','B']]

#I'm able to concatenate
df['C']=df['A']+df['B']

    NAME    A   B   C
  0 JOHN    [1, 2, 3]   [2, 3, 4]   [1, 2, 3, 2, 3, 4]
  1 MARY    [2, 3, 4]   [3, 4, 5]   [2, 3, 4, 3, 4, 5]
  2 CHARLIE [3, 4, 5]   [4, 5, 6]   [3, 4, 5, 4, 5, 6]
</code></pre>

<p>Any way to find the differences?</p>

<pre><code>df['C']=df['A']-df['B']
</code></pre>

<p>I know we can use <code>df.apply</code> to a function but row by row processing will run slow since I have around 400K rows. I'm looking for a straight forward method like </p>

<pre><code>df['C']=df['A']+df['B']
</code></pre>

<p>For a set difference,</p>

<pre><code>df['A'].map(set) - df['B'].map(set)
</code></pre>

python list pandas dataframe