Exclude Item from Web-Scraped Loop
<p>Suppose I have the following <code>html</code>:</p>

<pre><code>&lt;h4&gt;
     &lt;a href="http://www.google.com"&gt;Google&lt;/a&gt;
&lt;/h4&gt;
&lt;h4&gt;Random Text&lt;/h4&gt;
</code></pre>

<p>I am able to identify all <code>h4</code> headings via a loop such as:</p>

<pre><code>for url in soup.findAll("h4")
    print(url.get_text())
</code></pre>

<p>And that works well except it includes the "random text" element of the <code>h4</code> heading.  Is it possible to programmatically remove occurrences of <code>h4</code> headings that do not meet a certain criteria - for example, those that don't contain a link?</p>

<p>Sure, you can go with a straightforward approach, simply filtering the headings:</p>

<pre><code>for url in soup.find_all("h4")
    if not url.a:  # "url.a" is a shortcut to "url.find('a')"
        continue
    print(url.get_text())
</code></pre>

<p>Or, a better way would be to filter them with a <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/#a-function" rel="nofollow">function</a>:</p>

<pre><code>for url in soup.find_all(lambda tag: tag.name == "h4" and tag.a):
    print(url.get_text())
</code></pre>

<p>Or, even better, go straight to the <code>a</code> elements:</p>

<pre><code>for url in soup.select("h4 &gt; a"):
    print(url.get_text())
</code></pre>

<p><code>h4 &gt; a</code> here is a <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/#css-selectors" rel="nofollow">CSS selector</a> that would match <code>a</code> elements that are direct children of <code>h4</code> tags.</p>

python web-scraping beautifulsoup