How to check values of column in one dataframe available or not in column of another dataframe?
<p>I have two dataframes-</p>

<pre><code>df1_data = {'sym1' :{0:'abc a01',1:'pqr q02',2:'xyz y03',3:'mno o12',4:'lmn l45'}}
df1 = pd.DataFrame(df1_data)
print df1

df2_data = {'sym2' :{0:'abc a01',1:'xxx p0',2:'xyz y03',3:'mno o12',4:'lmn l45',5:'rrr r1',6:'kkk k3'}}
df2 = pd.DataFrame(df2_data)
print df2
</code></pre>

<p><strong>output-</strong></p>

<pre><code>      sym1
0  abc a01
1  pqr q02
2  xyz y03
3  mno o12
4  lmn l45
      sym2
0  abc a01
1   xxx p0
2  xyz y03
3  mno o12
4  lmn l45
5   rrr r1
6   kkk k3
</code></pre>

<p>I want to check sym2 column values available or not in df2 dataframes sym1 column. If symbols in sym2 column are not available then I want list of that symbols which are not available in sym1 column. If all symbols are available then list must be empty.</p>

<p><strong>Expected Result-</strong></p>

<pre><code>list -&gt; ['xxx p0','rrr r1','kkk k3']
</code></pre>

<p>Here is another, bit faster, solution:</p>

<pre><code>In [54]: df2.set_index('sym2').index.difference(df1.set_index('sym1').index).values
Out[54]: array(['kkk k3', 'rrr r1', 'xxx p0'], dtype=object)
</code></pre>

<p>or as vanilla Python list:</p>

<pre><code>In [74]: df2.set_index('sym2').index.difference(df1.set_index('sym1').index).values.tolist()
Out[74]: ['kkk k3', 'rrr r1', 'xxx p0']
</code></pre>

<p>Timings for 700K and 500K DFs:</p>

<pre><code>In [55]: df1 = pd.concat([df1] * 10**5, ignore_index=True)

In [57]: df2 = pd.concat([df2] * 10**5, ignore_index=True)

In [58]: df1.shape
Out[58]: (500000, 1)

In [59]: df2.shape
Out[59]: (700000, 1)

In [67]: %timeit df2.set_index('sym2').index.difference(df1.set_index('sym1').index).values
10 loops, best of 3: 123 ms per loop

In [68]: %timeit df2.ix[~df2.sym2.isin(df1.sym1), 'sym2']
1 loop, best of 3: 216 ms per loop

In [72]: %timeit df2.set_index('sym2').index.difference(df1.set_index('sym1').index).values.tolist()
10 loops, best of 3: 123 ms per loop
</code></pre>

python pandas dataframe