Finding duplicate rows python
<p>I have <code>timestamp</code> and <code>id</code> variables in my dataframe (<code>df</code>)</p>

<pre><code>timestamp                    id
2016-06-09 8:33:37           a1  
2016-06-09 8:33:37           a1  
2016-06-09 8:33:38           a1
2016-06-09 8:33:39           a1
2016-06-09 8:33:39           a1
2016-06-09 8:33:37           b1
2016-06-09 8:33:38           b1
</code></pre>

<p>Each <code>id</code> can't have two timestamps. I have to print these duplicate timestamps for each <code>id</code>. In my above case, the output should be for rows 1,2,4,5</p>

<p>The following code will give the duplicate <code>timestamp</code></p>

<pre><code>set([x for x in df['timestamp'] if df['timestamp'].count(x) &gt; 1])
</code></pre>

<p>How to consider <code>id</code> along with <code>timestamp</code> to have the duplicate rows?</p>

<p>You can use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html" rel="nofollow"><code>groupby</code></a> and get mask of all duplicates values per group by <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.duplicated.html" rel="nofollow"><code>Series.duplicated</code></a>. Last use <a href="http://pandas.pydata.org/pandas-docs/stable/indexing.html#boolean-indexing" rel="nofollow"><code>boolean indexing</code></a>:</p>

<pre><code>print (df.groupby(['id'])['timestamp'].apply(lambda x: x.duplicated(keep=False)))
0     True
1     True
2    False
3     True
4     True
5    False
6    False
Name: timestamp, dtype: bool

print (df[df.groupby(['id'])['timestamp'].apply(lambda x: x.duplicated(keep=False))])
            timestamp  id
0 2016-06-09 08:33:37  a1
1 2016-06-09 08:33:37  a1
3 2016-06-09 08:33:39  a1
4 2016-06-09 08:33:39  a1
</code></pre>

python pandas timestamp duplicates