Adding a column which increment for every index which meets a criteria on another column
<p>I am trying to generate a column from a DataFrame to base my grouping on. I know that every NaN column under a non NaN one belong to the same group. So I wrote this loop (cf below) but I was wondering if there was a more pandas/pythonic way to write it with apply or a comprehension list.</p>

<pre><code>import pandas

&gt;&gt;&gt; DF = pandas.DataFrame([134, None, None, None, 129374, None, None, 12, None],
                      columns=['Val'])
&gt;&gt;&gt; a = [0]
&gt;&gt;&gt; for i in DF['Val']:
        if i &gt; 1:
            a.append(a[-1] + 1)
        else:
            a.append(a[-1])
&gt;&gt;&gt; a.pop(0)  # remove 1st 0 which does not correspond to any rows
&gt;&gt;&gt; DF['Group'] = a
&gt;&gt;&gt; DF
        Val  Group
0     134.0      1
1       NaN      1
2       NaN      1
3       NaN      1
4  129374.0      2
5       NaN      2
6       NaN      2
7      12.0      3
8       NaN      3
</code></pre>

<p>Use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.notnull.html" rel="nofollow"><code>pd.notnull</code></a> to identify non-NaN values. Then use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.cumsum.html" rel="nofollow"><code>cumsum</code></a> to create the <code>Group</code> column:</p>

<pre><code>import pandas as pd

df = pd.DataFrame([134, None, None, None, 129374, None, None, 12, None],
                  columns=['Val'])
df['Group'] = pd.notnull(df['Val']).cumsum()
print(df)
</code></pre>

<p>yields</p>

<pre><code>        Val  Group
0     134.0      1
1       NaN      1
2       NaN      1
3       NaN      1
4  129374.0      2
5       NaN      2
6       NaN      2
7      12.0      3
8       NaN      3
</code></pre>

python pandas