Efficiently creating a Pandas DataFrame Column that contains the instance number of a value in another column
<p>Assume that you have a Pandas column with the following information:</p>

<pre><code>&gt;&gt; df
     num
0    0
1    1
2    1
3    2
4    3
5    3
</code></pre>

<p>The column to the left of the num column is the index column.
I want to create an instance column that tells me what instance of num appears. This is the outcome that I want:</p>

<pre><code>&gt;&gt; df
     num  instance
0    0    1
1    1    1
2    1    2
3    2    1
4    3    1
5    3    2
</code></pre>

<p>Here's the code that I wrote to do this:</p>

<pre><code>&gt;&gt; my_list = []
&gt;&gt; for index, row in df.iterrows():
&gt;&gt;    my_list.append(df.loc[index,'num'])
&gt;&gt;    # The IF condition is done to prevent my_list from growing too big.
&gt;&gt;    if len(my_list)&gt;1:
&gt;&gt;        if my_list[len(my_list)-1] == my_list[len(my_list)-2]:
&gt;&gt;            del my_list[:len(my_list)-2]
&gt;&gt;    my_list['instance'] = len([element for element in my_list if \
                                 element == df.loc[index,'num'])
</code></pre>

<p>This code works perfectly for small DataFrames, but it takes an exorbitantly long amount of time to complete when the num column consists of several million lines. Is there a way of creating the instance column in the manner that I'm thinking without using .iterrows()?</p>

<p>try this:</p>

<pre><code>In [11]: df['instance'] = df.groupby('num').cumcount()+1

In [12]: df
Out[12]:
   num  instance
0    0         1
1    1         1
2    1         2
3    2         1
4    3         1
5    3         2
</code></pre>

python pandas dataframe