How Can I Detect Gaps and Consecutive Periods In A Time Series In Pandas
<p>I have a pandas Dataframe that is indexed by Date.  I would like to select all consecutive gaps by period and all consecutive days by Period.    How can I do this?</p>

<p><strong>Example of Dataframe with No Columns but a Date Index:</strong></p>

<pre><code>In [29]: import pandas as pd

In [30]: dates = pd.to_datetime(['2016-09-19 10:23:03', '2016-08-03 10:53:39','2016-09-05 11:11:30', '2016-09-05 11:10:46','2016-09-05 10:53:39'])

In [31]: ts = pd.DataFrame(index=dates)
</code></pre>

<p>As you can see there is a <em>gap from 2016-08-03 and 2016-09-19</em>.  How do I detect these so I can create descriptive statistics, i.e.  40 gaps, with median gap duration of "x", etc.  Also, I can see that <em>2016-09-05 and 2016-09-06 is a two day range</em>.  How I can detect these and also print descriptive stats?</p>

<p>Ideally the result would be returned as another Dataframe in each case since I want use other columns in the Dataframe to groupby. </p>

<p>here's something to get started:</p>

<pre><code>df = pd.DataFrame(np.ones(5),columns = ['ones'])
df.index = pd.DatetimeIndex(['2016-09-19 10:23:03', '2016-08-03 10:53:39', '2016-09-05 11:11:30', '2016-09-05 11:10:46', '2016-09-06 10:53:39'])
daily_rng = pd.date_range('2016-08-03 00:00:00', periods=48, freq='D')
daily_rng = daily_rng.append(df.index)
daily_rng = sorted(daily_rng)
df =  df.reindex(daily_rng).fillna(0)
df = df.astype(int)
df['ones'] = df.cumsum()
</code></pre>

<p>The cumsum() creates a grouping variable on 'ones' partitioning your data at the points your provided. If you print df to say a spreadsheet it will make sense:</p>

<pre><code>print df.head()

                     ones
2016-08-03 00:00:00     0
2016-08-03 10:53:39     1
2016-08-04 00:00:00     1
2016-08-05 00:00:00     1
2016-08-06 00:00:00     1

print df.tail()
                     ones
2016-09-16 00:00:00     4
2016-09-17 00:00:00     4
2016-09-18 00:00:00     4
2016-09-19 00:00:00     4
2016-09-19 10:23:03     5
</code></pre>

<p>now to complete:</p>

<pre><code>df = df.reset_index()
df = df.groupby(['ones']).aggregate({'ones':{'gaps':'count'},'index':{'first_spotted':'min'}})
df.columns = df.columns.droplevel()
</code></pre>

<p>which gives:</p>

<pre><code>              first_time  gaps
ones                          
0    2016-08-03 00:00:00     1
1    2016-08-03 10:53:39    34
2    2016-09-05 11:10:46     1
3    2016-09-05 11:11:30     2
4    2016-09-06 10:53:39    14
5    2016-09-19 10:23:03     1
</code></pre>

python pandas