Scraping Javascript using Selenium via Python
<p>I'm trying to scrape javascript data from a site. Currently I'm given myself the challenge of trying to scrape the number of Followers from <a href="http://freelegalconsultancy.blogspot.co.uk/" rel="nofollow">this website</a>. Here's my code so far: </p>

<pre><code>import os
from selenium import webdriver
import time

chromedriver = "/Users/INSERT USERNAME/Desktop/chromedriver"
os.environ["webdriver.chrome.driver"] = chromedriver
driver = webdriver.Chrome(chromedriver)
driver.get("http://freelegalconsultancy.blogspot.co.uk/")
time.sleep(5)

title = driver.find_element_by_class_name
print title
</code></pre>

<p>As you can see, I've got a chromedriver file located on my desktop. When I execute the code, I get the following result:</p>

<pre><code>&lt;bound method WebDriver.find_element_by_class_name of &lt;selenium.webdriver.chrome.webdriver.WebDriver (session="dd9e5d3f429bc2810c30ebe7067e4e22")&gt;&gt;
</code></pre>

<p>I tried iterating into this with a for loop but it returned an error. Does anyone know how I can get the Javascript data and ultimately get the number of followers? </p>

<p><strong>EDIT:</strong></p>

<p>So as per request, I have changed my code to this:</p>

<pre><code>import os
from selenium import webdriver
import time

chromedriver = "/Users/INSERT USERNAME/Desktop/chromedriver"
os.environ["webdriver.chrome.driver"] = chromedriver
driver = webdriver.Chrome(chromedriver)
driver.get("http://freelegalconsultancy.blogspot.co.uk/")
time.sleep(5)

title = driver.find_element_by_class_name("member-title")
print title
</code></pre>

<p>But I now get this error:</p>

<pre><code>Traceback (most recent call last):
  File "C:\Users\INSERT USERNAME\Desktop\blogger_v.1.py", line 11, in &lt;module&gt;
    title = driver.find_element_by_class_name("member-title")
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 413, in find_element_by_class_name
    return self.find_element(by=By.CLASS_NAME, value=name)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 752, in find_element
    'value': value})['value']
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"class name","selector":"member-title"}
  (Session info: chrome=53.0.2785.143)
  (Driver info: chromedriver=2.24.417431 (9aea000394714d2fbb20850021f6204f2256b9cf),platform=Windows NT 6.1.7601 SP1 x86_64)
</code></pre>

<p>Any ideas on how I can get around it? </p>

<p><strong>EDIT:</strong></p>

<p>So I've changed my code to this:</p>

<pre><code>import os
from selenium import webdriver
import time

chromedriver = "/Users/INSERT USERNAME/Desktop/chromedriver"
os.environ["webdriver.chrome.driver"] = chromedriver
driver = webdriver.Chrome(chromedriver)
driver.get("http://freelegalconsultancy.blogspot.co.uk/")
time.sleep(5)

title = driver.find_element_by_class_name("item-title")
print title
</code></pre>

<p>And I get this result:</p>

<pre><code>&lt;selenium.webdriver.remote.webelement.WebElement (session="5fe8fb966edd26fdf808da07f99d4109", element="0.9924860218635834-1")&gt;
</code></pre>

<p>How would I go about just printing all the javascript? Is this even possible?</p>

<p>You need to provide the class name you're looking for as a parameter.</p>

<p><code>title = driver.find_element_by_class_name("TheNameOfTheClass")</code></p>

javascript python selenium