Python 2.7 BeautifulSoup , email scraping
<p>Hope you are all well. I'm new in Python and using python 2.7.</p>

<p>I'm trying to extract only the mailto from this public website business directory: <a href="http://www.tecomdirectory.com/companies.php?segment=&amp;activity=&amp;search=category&amp;submit=Search" rel="nofollow">http://www.tecomdirectory.com/companies.php?segment=&amp;activity=&amp;search=category&amp;submit=Search</a><br>
the mails i'm looking for are the emails mentioned in every widget from a-z in the full directory. This directory does not have an API unfortunately.
I'm using BeautifulSoup, but with no success so far.<br>
here is mycode:</p>

<pre><code>import urllib
from bs4 import BeautifulSoup
website = raw_input("Type website here:&gt;\n")
html = urllib.urlopen('http://'+ website).read()
soup = BeautifulSoup(html)

tags = soup('a') 

for tag in tags:
    print tag.get('href', None)
</code></pre>

<p>what i get is just the website of the actual website , like <a href="http://www.tecomdirectory.com" rel="nofollow">http://www.tecomdirectory.com</a> with other href rather then the mailto or websites in the widgets. i also tried replacing soup('a') with soup ('target'), but no luck! Can anybody help me please?</p>

<p>You cannot just find every anchor, you need to specifically look for "mailto:" in the href, you can use a css selector <code>a[href^=mailto:]</code> which finds <em>anchor</em> tags that have a <em>href</em>  starting with <code>mailto:</code>:</p>

<pre><code>import requests

soup  = BeautifulSoup(requests.get("http://www.tecomdirectory.com/companies.php?segment=&amp;activity=&amp;search=category&amp;submit=Search").content)

print([a["href"] for a in soup.select("a[href^=mailto:]")])
</code></pre>

<p>Or extract the text:</p>

<pre><code>print([a.text for a in soup.select("a[href^=mailto:]")])
</code></pre>

<p>Using <code>find_all("a")</code> you would need to use a regex to achieve the same:</p>

<pre><code>import re

find_all("a", href=re.compile(r"^mailto:"))
</code></pre>

python python-2.7 web-scraping beautifulsoup