Unable to find text although it is present
<p>I am learning web-scrapping with Python. I wrote code to retrieve company names form one of Indian yellow pages</p>

<pre><code>r =  requests.get("http://xyzxyz", headers={'User-Agent' : "Magic Browser"})
soup= BeautifulSoup(r.content,"html.parser")

for link in soup.findAll("div", {"class" : "col-sm-5"}):
    coLink = link.find("span" , {"class" : "jcn"})
    companyName = coLink.find("a").text
</code></pre>

<p>I am getting error  " AttributeError: 'NoneType' object has no attribute 'find'". I know we get this error if object does not find anything. However if when I print(coLink), It gives following link inside every span class</p>

<pre><code>&lt;span class="jcn"&gt;&lt;a href="http://xyz/Kolkata/Sunrise-International-&amp;lt;near&amp;gt;-B-R-B-Basu-Road-/033P3001041_BZDET?xid=S29sa2F0YSBUYXBlciBSb2xsZXIgQmVhcmluZyBEZWFsZXJz" onclick="_ct('clntnm', 'lspg');" title="Sunrise International in , Kolkata"&gt;Sunrise International&lt;/a&gt;
&lt;/span&gt;
&lt;span class="jcn"&gt;&lt;a href="http://xyz/Kolkata/Shree-Shakti-Vyapaar-PVT-LTD/033P6001995_BZDET?xid=S29sa2F0YSBUYXBlciBSb2xsZXIgQmVhcmluZyBEZWFsZXJz" onclick="_ct('clntnm', 'lspg');" title="Shree Shakti Vyapaar PVT LTD in , Kolkata"&gt;Shree Shakti Vyapaar PVT LTD&lt;/a&gt;
&lt;/span&gt;
</code></pre>

<p>Can you please help how to get text of company?</p>

<p>Not explaining the current problem, but providing an alternative solution - you can use a <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/#css-selectors" rel="nofollow">CSS selector</a> that would match the desired <code>a</code> elements:</p>

<pre><code>for link in soup.select(".col-sm-5 .jcn a"):
    print(link.get_text())
</code></pre>

<p>This way you would not get any errors in case there is a <code>span</code> with no <code>a</code> link inside.</p>

<p>Note that <code>col-sm-5</code> is a terrible class name to be using to locate elements - it is UI/layout-specific and has a high chance of being changed.</p>

python web-scraping beautifulsoup find