Calculate cumulative p_value hourly in pandas
<p>I am wondering if there is a way to calculate the cumulative p_value for each hour of data in a dataframe. For example if you have 24 hours of data there would be 24 measurements of p_value, but they would be cumulative for all hours before the current hour.</p>

<p>I have been able to get the p_value for each hour by grouping my data by hour and then applying an agg_func that I wrote to calculate all of the relevant statistics necessary to calculate p. However, this approach does not produce a cumulative result, only the p for each individual hour. </p>

<p>Given a df with columns id, ts (as unix timestamp), ab_group, result. I ran the following code to compute p_values on the hour. </p>

<pre><code>df['time'] = pd.to_datetime(df.ts, unit='s').values.astype('&lt;m8[h]')

def calc_p(group):
    df_old_len = len(group[group.ab_group == 0])
    df_new_len = len(group[group.ab_group == 1])
    ctr_old = float(len(group[(group.ab_group == 0) &amp; (df.result == 1)]))/ df_old_len
    ctr_new = float(len(group[(group.ab_group == 1) &amp; (df.converted == 1)]))/ df_new_len
    nobs_old = df_old_len
    nobs_new = df_new_len
    z_score, p_val, null = z_test.z_test(ctr_old, ctr_new, nobs_old, nobs_new, effect_size=0.001)
    return p_val

grouped = df.groupby(by='time').agg(calc_p)
</code></pre>

<p>N.B. z_test is my own module containing an implementation of a z_test. </p>

<p>Any advice on how to modify this for a cumulative p is much appreciated.</p>

<p>So i came up with a workaround on my own for this one.</p>

<p>What I came up with was modifying <code>calc_p()</code> such that it utilized global variables and thus could use updated values each time it was called by the aggfunc. Below is the edited code:</p>

<pre><code>def calc_p(group):
    global df_old_len, df_new_len, clicks_old, clicks_new
    clicks_old += len(group[(group.landing_page == 'old_page') &amp; (group.converted == 1)])
    clicks_new += len(group[(group.landing_page == 'new_page') &amp; (group.converted == 1)])
    df_old_len += len(group[group.landing_page == 'old_page'])
    df_new_len += len(group[group.landing_page == 'new_page'])
    ctr_old = float(clicks_old)/df_old_len
    ctr_new = float(clicks_new)/df_new_len
    z_score, p_val, null = z_test.z_test(ctr_old, ctr_new, df_old_len, df_new_len, effect_size=0.001)
    return p_val

# Initialize global values to 0 for cumulative calc_p
df_old_len = 0
df_new_len = 0
clicks_old = 0
clicks_new = 0

grouped = df.groupby(by='time').agg(calc_p)
</code></pre>

python pandas grouping p-value