Scraping HTML data from website in Python
<p>I'm trying to scrape certain pieces of HTML data from certain websites, but I can't seem to scrape the parts I want. For instance I set myself the challenge of scraping the number of followers from <a href="http://freelegalconsultancy.blogspot.co.uk/" rel="nofollow">this blog</a>, but I can't seem to do so. </p>

<p>I've tried using urllib, request, beautifulsoup as well as <a href="https://www.jamapi.xyz/" rel="nofollow">Jam API</a>. </p>

<p>Here's what my code looks like at the moment:</p>

<pre><code>from bs4 import BeautifulSoup
from urllib import urlopen
import json
import urllib2

html = urlopen('http://freelegalconsultancy.blogspot.co.uk/')
soup = BeautifulSoup(html, "lxml")
print soup
</code></pre>

<p>How would I go about pulling the number of followers in this instace?</p>

<p>You can't grab the followers as it's a widget loaded by javascript.  You need to grab parts of the html by css class or id or by the element.</p>

<p>E.g:</p>

<pre><code>from bs4 import BeautifulSoup
from urllib import urlopen

html = urlopen('http://freelegalconsultancy.blogspot.co.uk/')
soup = BeautifulSoup(html)

assert soup.h1.string == '\nLAW FOR ALL-M.MURALI MOHAN\n'
</code></pre>

python html