Changing the value of groups with few members in a pandas data frame
<p>I have a data frame which represent different classes with their values. for example:</p>

<pre><code>df=pd.DataFrame(
 {'label':['a','a','b','a','b','b','a','c','c','d','e','c'],
'date':[1,2,3,4,3,7,12,18,11,2,5,3],'value':np.random.randn(12)})
</code></pre>

<p>I want to choose the labels with values_counts less than a specific threshold and then put them into one class i.e. label them as for example 'zero'. </p>

<p>This is my attemp:</p>

<pre><code>value_count=df.label.value_counts()
threshold = 3
for index in value_count[value_count.values&lt;=threshold].index:
    df.label[df.label==index]='zero'
</code></pre>

<p>Is there a better way to do this?</p>

<p>You can use groupby.transform to get the value counts aligned with the original index, then use it as a boolean index:</p>

<pre><code>df.loc[df.groupby('label')['label'].transform('count') &lt;= threshold, 'label'] = 'zero'

df
Out: 
    date label     value
0      1     a -0.587957
1      2     a  0.341551
2      3  zero  0.516933
3      4     a  0.234042
4      3  zero -0.206185
5      7  zero  0.840724
6     12     a -0.728868
7     18  zero  0.111260
8     11  zero -0.471337
9      2  zero  0.030803
10     5  zero  1.012638
11     3  zero -1.233750
</code></pre>

<p>Here are my timings:</p>

<pre><code>df = pd.concat([df]*10**4)

%timeit df.groupby('label')['label'].transform('count') &lt;= threshold
100 loops, best of 3: 7.86 ms per loop

%%timeit 
value_count=df.label.value_counts()
df['label'].isin(value_count[value_count.values&lt;=threshold].index)
100 loops, best of 3: 9.24 ms per loop
</code></pre>

python pandas