Python pandas check if the last element of a list in a cell contains specific string
<pre><code>my dataframe df:

index                        url
1           [{'url': 'http://bhandarkarscollegekdp.org/'}]
2             [{'url': 'http://cateringinyourhome.com/'}]
3                                                     NaN
4                  [{'url': 'http://muddyjunction.com/'}]
5                       [{'url': 'http://ecskouhou.jp/'}]
6                     [{'url': 'http://andersrice.com/'}]
7       [{'url': 'http://durager.cz/'}, {'url': 'http:andersrice.com'}]
8            [{'url': 'http://milenijum-osiguranje.rs/'}]
9       [{'url': 'http://form-kind.org/'}, {'url': 'https://osiguranje'},{'url': 'http://beseka.com.tr'}]
</code></pre>

<p>I would like to select the rows if the last item in the list of the row of url column contains 'https', while skipping missing values.</p>

<p>My current script </p>

<pre><code>df[df['url'].str[-1].str.contains('https',na=False)]
</code></pre>

<p>returns False values for all the rows while some of them actually contains https. </p>

<p>Can anybody help with it?</p>

<p>I think you can first replace <code>NaN</code> to <code>empty url</code> and then use <code>apply</code>:</p>

<pre><code>df = pd.DataFrame({'url':[[{'url': 'http://bhandarkarscollegekdp.org/'}],
                          np.nan,
                         [{'url': 'http://cateringinyourhome.com/'}],  
                         [{'url': 'http://durager.cz/'}, {'url': 'https:andersrice.com'}]]},
                  index=[1,2,3,4])

print (df)
                                                 url
1     [{'url': 'http://bhandarkarscollegekdp.org/'}]
2                                                NaN
3        [{'url': 'http://cateringinyourhome.com/'}]
4  [{'url': 'http://durager.cz/'}, {'url': 'https...
</code></pre>

<hr>

<pre><code>df.loc[df.url.isnull(), 'url'] = [[{'url':''}]]
print (df)
                                                 url
1     [{'url': 'http://bhandarkarscollegekdp.org/'}]
2                                      [{'url': ''}]
3        [{'url': 'http://cateringinyourhome.com/'}]
4  [{'url': 'http://durager.cz/'}, {'url': 'https...

print (df.url.apply(lambda x: 'https' in x[-1]['url']))
1    False
2    False
3    False
4     True
Name: url, dtype: bool
</code></pre>

<p>First solution:</p>

<pre><code>df.loc[df.url.notnull(), 'a'] = 
df.loc[df.url.notnull(), 'url'].apply(lambda x: 'https' in x[-1]['url'])

df.a.fillna(False, inplace=True)
print (df)
                                                 url      a
1     [{'url': 'http://bhandarkarscollegekdp.org/'}]  False
2                                                NaN  False
3        [{'url': 'http://cateringinyourhome.com/'}]  False
4  [{'url': 'http://durager.cz/'}, {'url': 'https...   True
</code></pre>

python loops pandas contain