Pandas, split dataframe by monotonic increase of column value
<p>I have a gigantic dataframe with a datetime type column called time, and another float type column called dist, the data frame is sorted based on time, and dist already. 
I want to split the dataframe into several dataframes base on monotonic increase of dist.</p>

<p>Split</p>

<pre><code>   dt                    dist
0  20160811 11:10        1.0
1  20160811 11:15        1.4
2  20160811 12:15        1.8
3  20160811 12:32        0.6
4  20160811 12:34        0.8
5  20160811 14:38        0.2
</code></pre>

<p>into</p>

<pre><code>   dt                    dist
0  20160811 11:10        1.0
1  20160811 11:15        1.4
2  20160811 12:15        1.8

   dt                    dist
0  20160811 12:32        0.6
1  20160811 12:34        0.8

   dt                    dist
0  20160811 14:38        0.2
</code></pre>

<p>you can do it using <a href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.split.html" rel="nofollow">np.split()</a> method:</p>

<pre><code>In [92]: df
Out[92]:
                   dt  dist
0 2016-08-11 11:10:00   1.0
1 2016-08-11 11:15:00   1.4
2 2016-08-11 12:15:00   1.8
3 2016-08-11 12:32:00   0.6
4 2016-08-11 12:34:00   0.8
5 2016-08-11 14:38:00   0.2

In [93]: dfs = np.split(df, df[df.dist.diff().fillna(0) &lt; 0].index)

In [94]: [print(x) for x in dfs]
                   dt  dist
0 2016-08-11 11:10:00   1.0
1 2016-08-11 11:15:00   1.4
2 2016-08-11 12:15:00   1.8
                   dt  dist
3 2016-08-11 12:32:00   0.6
4 2016-08-11 12:34:00   0.8
                   dt  dist
5 2016-08-11 14:38:00   0.2
Out[94]: [None, None, None]
</code></pre>

<p>Explanation:</p>

<pre><code>In [97]: df.dist.diff().fillna(0) &lt; 0
Out[97]:
0    False
1    False
2    False
3     True
4    False
5     True
Name: dist, dtype: bool

In [98]: df[df.dist.diff().fillna(0) &lt; 0]
Out[98]:
                   dt  dist
3 2016-08-11 12:32:00   0.6
5 2016-08-11 14:38:00   0.2

In [99]: df[df.dist.diff().fillna(0) &lt; 0].index
Out[99]: Int64Index([3, 5], dtype='int64')
</code></pre>

python pandas numpy dataframe