How to remove rows with duplicates in pandas dataframe?
<p>Having a dataframe which contains duplicate values in two columns (<code>A</code> and <code>B</code>):</p>

<pre><code>A B
1 2
2 3
4 5
7 6
5 8
</code></pre>

<p>I want to remove duplicates so that only unique values remain:</p>

<pre><code>A B
1 2
4 5
7 6
</code></pre>

<p>This command does not provide what I want:</p>

<pre><code>df.drop_duplicates(subset=['A','B'], keep='first')
</code></pre>

<p>Any idea how to do this?</p>

<p>You can use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.stack.html" rel="nofollow"><code>stack</code></a> with <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.unstack.html" rel="nofollow"><code>unstack</code></a>:</p>

<pre><code>print (df.stack().drop_duplicates().unstack().dropna().astype(int))
   A  B
0  1  2
2  4  5
3  7  6
</code></pre>

<p>Solution with <a href="http://pandas.pydata.org/pandas-docs/stable/indexing.html#boolean-indexing" rel="nofollow"><code>boolean indexing</code></a>:</p>

<pre><code>print (df[~df.stack().duplicated().unstack().any(1)])
   A  B
0  1  2
2  4  5
3  7  6
</code></pre>

python pandas indexing duplicates multiple-columns