apply 1-to-group tranformations in pandas - python
<p>I have a dataframe like the following</p>

<pre><code>import pandas as pd
df = pd.DataFrame({"id": ["a", "b", "c", "d"], "v": [1,2,3,4], "type": ["X", "Y", "Y", "Y"]}).set_index("id")

print(df)
</code></pre>

<p>which yields:</p>

<pre><code>   type  v
id        
a     X  1
b     Y  2
c     Y  3
d     Y  4
</code></pre>

<p>and I want to subtract the mean fromeach value <strong>BY GROUP</strong>. after the operation I still want to have <strong>my SINGLE values</strong>. in other words I want to have</p>

<pre><code>   type  v
id        
a     X  0
b     Y  -1
c     Y  0
d     Y  1
</code></pre>

<p>so, the very useful <code>tranform</code> function applied to a <code>groupby</code> object (as detailed here <a href="http://pandas.pydata.org/pandas-docs/stable/groupby.html" rel="nofollow">http://pandas.pydata.org/pandas-docs/stable/groupby.html</a>) is not very useful in my case and I was wondering how to approach the problem.</p>

<p>this is not to say I cannot solve this problem when dealing with simple functions <em>(division/subtract => I can use an auxiliary datafrmae resulting from a groupy and then cross the results but when dealing with more complex stuff this is no longer the case)</em></p>

<p>Transform will actually get you what you want (if I understand correctly):</p>

<pre><code>df['v'] = df['v'] - df.groupby('type')['v'].transform('mean')
</code></pre>

<p>Transform calculates the applied function by group, but broadcasts the result on the original index.</p>

<hr>

<p><strong>Edit</strong>: timing comparisons</p>

<pre><code>%timeit df.groupby("type")['v'].apply(lambda x: x-x.mean())
100 loops, best of 3: 2.95 ms per loop

%timeit df['v'] - df.groupby('type')['v'].transform('mean')
1000 loops, best of 3: 922 Âµs per loop
</code></pre>

python function pandas grouping