Pandas Slow. Want first occurrence in DataFrame
<p>I have a DataFrame of <code>people</code>. One of the columns in this DataFrame is a <code>place_id</code>. I also have a DataFrame of places, where one of the columns is <code>place_id</code> and another is <code>weather</code>. For every person, I am trying to find the corresponding weather. Importantly, many people have the same <code>place_id</code>s. </p>

<p>Currently, my setup is this:</p>

<pre><code>def place_id_to_weather(pid):
    return place_df[place_df['place_id'] == pid]['weather'].item() 

person_df['weather'] = person_df['place_id'].map(place_id_to_weather)`
</code></pre>

<p>But this is untenably slow. I would like to speed this up. I suspect that I could achieve a speedup like this:</p>

<p>Instead of returning <code>place_df[...].item()</code>, which does a search for <code>place_id == pid</code> for that entire column and returns a series, and then grabbing the first item in that series, I really just want to curtail the search in <code>place_df</code> after the first match <code>place_df['place_id']==pid</code> has been found. After that, I don't need to search any further. How do I limit the search to first occurrences only?</p>

<p>Are there other methods I could use to achieve a speedup here? Some kind of join-type method?</p>

<p>You can use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.merge.html" rel="nofollow"><code>merge</code></a> to do the operation :</p>

<pre><code>people = pd.DataFrame([['bob', 1], ['alice', 2], ['john', 3], ['paul', 2]], columns=['name', 'place'])

#    name  place
#0    bob      1
#1  alice      2
#2   john      3
#3   paul      2

weather = pd.DataFrame([[1, 'sun'], [2, 'rain'], [3, 'snow'], [1, 'rain']], columns=['place', 'weather'])

#   place weather
#0      1     sun
#1      2    rain
#2      3    snow
#3      1    rain

pd.merge(people, weather, on='place')

#    name  place weather
#0    bob      1     sun
#1    bob      1    rain
#2  alice      2    rain
#3   paul      2    rain
#4   john      3    snow
</code></pre>

<p>In case you have several  weather for the same place, you may want to use <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.drop_duplicates.html" rel="nofollow"><code>drop_duplicates</code></a>, then you have the following result :</p>

<pre><code>pd.merge(people, weather, on='place').drop_duplicates(subset=['name', 'place'])

#    name  place weather
#0    bob      1     sun
#2  alice      2    rain
#3   paul      2    rain
#4   john      3    snow
</code></pre>

python pandas