Pandas aggregate list in resample/groupby
<p>I have a dataframe in which each instance has a timestamp, an id and a list of numbers as follows: </p>

<pre><code>timestamp           | id | lists
----------------------------------
2016-01-01 00:00:00 | 1  | [2, 10]
2016-01-01 05:00:00 | 1  | [9, 10, 3, 5]
2016-01-01 10:00:00 | 1  | [1, 10, 5]
2016-01-02 01:00:00 | 1  | [2, 6, 7]
2016-01-02 04:00:00 | 1  | [2, 6]
2016-01-01 02:00:00 | 2  | [0]
2016-01-01 08:00:00 | 2  | [10, 3, 2]
2016-01-01 14:00:00 | 2  | [0, 9, 3]
2016-01-02 03:00:00 | 2  | [0, 9, 2]
</code></pre>

<p>For each <em>id</em> I want to resample by day(and this is easy) and concatenate all the 
<em>lists</em> of the instances that happened in the same day. Resample + concat/sum does
not work because resample removes all non-numeric columns (<a href="http://stackoverflow.com/questions/34257069/resampling-pandas-dataframe-is-deleting-column">see here</a>)</p>

<p>I want to write something similar to this:</p>

<pre><code>daily_data = data.groupby('id').resample('1D').concatenate() # .concatenate() does not exist
</code></pre>

<p>Result desired:</p>

<pre><code>timestamp  | id | lists
----------------------------------
2016-01-01 | 1  | [2, 10, 9, 10, 3, 5, 1, 10, 5]
2016-01-02 | 1  | [2, 6, 7, 2, 6]
2016-01-01 | 2  | [0, 10, 3, 2]
2016-01-02 | 2  | [0, 9, 3, 0, 9, 2]
</code></pre>

<p>Here you can copy a script that generates the input I used for the description:</p>

<pre><code>import pandas as pd 
from random import randint

time = pd.to_datetime( ['2016-01-01 00:00:00', '2016-01-01 05:00:00', 
                        '2016-01-01 10:00:00', '2016-01-02 01:00:00', 
                        '2016-01-02 04:00:00', '2016-01-01 02:00:00', 
                        '2016-01-01 08:00:00', '2016-01-01 14:00:00',
                        '2016-01-02 03:00:00' ]
                      )

id_1 = [1] * 5
id_2 = [2] * 4

lists = [0] * 9
for i in range(9):
    l = [randint(0,10)  for _ in range(randint(1,5) ) ]
    l = list(set(l))
    lists[i] = l

data = {'timestamp': time, 'id': id_1 + id_2, 'lists': lists}

example = pd.DataFrame(data=data)
</code></pre>

<p>Bonus points if there is a way to optionally remove duplicates in the concatenated list.</p>

<p>for the unique count of each list item use list comprehension:</p>

<pre><code>a = [list(set(l)) for l in df.lists]
df.loc[:,'lists'] = a
</code></pre>

python pandas dataframe