How to replace the for loop by faster option in Python
<p>I have two data frames, which have approximately 10 000 rows. They are similar as a and b below but with more rows.</p>

<pre><code> a
 Out[9]: 
     end  start
 0   4.0      3
 1   5.5      5
 2   7.5      7
 3   9.5      9
 4  11.5     11
 5  15.0     14
 6  18.0     17
 7  21.0     20
 8  26.0     25
 9  31.0     30

 b
 Out[10]: 
        status
 moment       
 8.0         o
 10.0        o
 14.5        o
 16.0        o
 19.0        o
 27.0        o
 28.0        o
 30.5        o
 35.0        o
 40.0        o
 50.0        o
</code></pre>

<p>I have to find all moments in dataframe b which belong to between end and start in dataframe a.</p>

<p>I developed for loop for that and it works well with small dataframes.</p>

<pre><code> for r in a.index:    
     for k in b.index:
         if a.ix[r,'start'] &lt;k and k &lt;a.ix[r,'end']:
             b.ix[k,'status']='m' # replaces m to o if moment is in between start and end
</code></pre>

<p>Below you can see how for loop have replaced o -> m when moment is in between start and end.</p>

<pre><code> n [12]: b
 Out[12]: 
        status
 moment       
 8.0         o
 10.0        o
 14.5        m
 16.0        o
 19.0        o
 27.0        o
 28.0        o
 30.5        m
 35.0        o
 40.0        o
 50.0        o
</code></pre>

<p>When I try to use it with huge dataframes (more than 10 000 rows in dataframe) it cannot get results any more within reasonable time.</p>

<p>Do you have any ideas how to elaborate my for loop faster and suitable for longer dataframes?</p>

<p>Here is an option without a <code>for</code> loop, which doesn't avoid vector scan but is vectorized:</p>

<pre><code>b[b.index.map(lambda m: ((m &gt; a.start) &amp; (m &lt; a.end)).any())] = "m"

b

#   status
# moment    
# 8.0   o
# 10.0  o
# 14.5  m
# 16.0  o
# 19.0  o
# 27.0  o
# 28.0  o
# 30.5  m
# 35.0  o
# 40.0  o
# 50.0  o
</code></pre>

python pandas