Merge events temporally close to one another in numpy (pandas)
<p>I would like to simplify a list of start and stop times.  When the time between the stop of one and the start of another, I'd like to combine (rows).  The following is a simplification of my data and what I would like as output:</p>

<pre><code>import numpy as np
import pandas as pd
start_time = [  1,  7, 20, 22, 27, 35]
stop_time  = [  5,  9, 22, 26, 30, 40]
events = pd.DataFrame({'start_time': start_time, 'stop_time': stop_time})
allowable_gap = 2.0

desired_start_time = [  1, 20, 35]
desired_stop_time  = [  9, 30, 40]
desired_events = pd.DataFrame({'start_time':desired_start_time, 'stop_time':desired_stop_time})
</code></pre>

<p>I have no requirement that I must use Pandas.  However, I need to at least use numpy.  The number of events is in the order of 1e6.</p>

<p>Thanks for any implementations or guidance.  I know that part of my problem is that I don't "get" Pandas.</p>

<p>My usage likely isn't relevant to the solution.  As background, I'm collecting a large number of events and then plotting them using matplotlib.pyplot.  As the output is complex, the best format I have found is .svg.  IE usually renders fine but takes and exceptionally long time to do so and I hope to reduce the number of lines that it has to draw.  I would love to view time series in a better way but that is outside the scope of this question.</p>

<p>This solution uses <a href="http://pandas.pydata.org/pandas-docs/version/0.18.0/generated/pandas.DataFrame.iterrows.html" rel="nofollow">DataFrame.iterrows()</a> function.<br>
I made this assumption:</p>

<ul>
<li>start_time &lt;= stop_time for all events</li>
</ul>

<pre class="lang-python prettyprint-override"><code>import numpy as np
import pandas as pd
start_time = [  1,  7, 20, 22, 27, 35]
stop_time  = [  5,  9, 22, 26, 30, 40]
events = pd.DataFrame({'start_time': start_time, 'stop_time': stop_time})
allowable_gap = 2.0

desired_start_time = []
desired_stop_time  = []

start = None
end = None
for index, row in events.iterrows():
    if start == None and end == None:
        start = row['start_time']
        end = row['stop_time']
    else:
        if end + allowable_gap &gt;= row['start_time']:
            end = row['stop_time']
        else:
            desired_start_time.append(start)
            desired_stop_time.append(end)
            start = row['start_time']
            end = row['stop_time']
desired_start_time.append(start)
desired_stop_time.append(end)

print(desired_start_time)
print(desired_stop_time)
</code></pre>

<p></p>

<h3>Output:</h3>

<blockquote>
  <p>[1, 20, 35]<br> [9, 30, 40]</p>
</blockquote>

python pandas numpy svg