Pandas broadcasting values given logical condition in group by
<p>I have a data fram according to example below:</p>

<pre>
key1   key2     value1 
1      201501     NaN      
1      201502     NaN     
1      201503     201503      
1      201504     NaN      
2      201507     NaN
2      201508     NaN 
2      201509     NaN
3      201509     NaN 
3      201510     201509
3      201511     NaN
3      201512     NaN 
3      201513     NaN
</pre>

<p>and I want the following output... </p>

<pre>
key1   key2     value1     value2
1      201501     NaN      0
1      201502     NaN      0
1      201503     201503   1   
1      201504     NaN      1
2      201507     NaN      0        
2      201508     NaN      0
2      201509     NaN      0
3      201509     NaN      0
3      201510     201509   1
3      201511     NaN      1
3      201512     NaN      1
3      201601     NaN      1
</pre>

<p>The output is simply a binary flag that takes on the <em>value1</em> if it has a yyyymm-stamp in <em>value1</em> and then it keeps it for the reminder of its key1-group. In the rows preceeding it should be 0. If the <em>key1</em> only has <em>np.NaN</em> then it should be 0, like for <em>key1</em> = 2.</p>

<p>I have tried a version with an apply using lambda operator but its really slow. I was hoping someone could given me a tip on how to broadcast this using a more vectorized approach to save some execution time.</p>

<p>code for df below!</p>

<p>Many thanks in advance for time and input! </p>

<p>Best regards,</p>

<p>/swepab</p>

<pre><code>import numpy as np

df = pd.DataFrame({'key1' : [1,1,1,1,2,2,2,3,3,3,3,3]
              ,'key2' : [201501, 201502,201503,201504,201507,201508,201509,201509,201510,201511,201512,201601]
              ,'value1' : [np.nan,np.nan,'201503',np.nan,np.nan,np.nan,np.nan,np.nan,'201509',np.nan,np.nan,np.nan]
              ,'value2' : [0,0,1,1,0,0,0,0,1,1,1,1]})
</code></pre>

<p>You can do:</p>

<pre><code>df['value2'] = df.groupby('key1')['value1'].apply(lambda x: (~pd.isnull(x)).cumsum())

In [50]: df
Out[50]:
key1    key2  value1  value2
0      1  201501     NaN       0
1      1  201502     NaN       0
2      1  201503  201503       1
3      1  201504     NaN       1
4      2  201507     NaN       0
5      2  201508     NaN       0
6      2  201509     NaN       0
7      3  201509     NaN       0
8      3  201510  201509       1
9      3  201511     NaN       1
10     3  201512     NaN       1
11     3  201601     NaN       1
</code></pre>

python pandas transform