Scraping a specific table in selenium
<p>I am trying to scrape a table found inside a div on a page. </p>

<p>Basically here's my attempt so far:</p>

<pre><code># NOTE: Download the chromedriver driver
# Then move exe file on C:\Python27\Scripts
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
import sys

driver = webdriver.Chrome()
driver.implicitly_wait(10)

URL_start = "http://www.google.us/trends/explore?"
date = '&amp;date=today%203-m' # Last 90 days
location = "&amp;geo=US"
symbol = sys.argv[1]
query = 'q='+symbol
URL = URL_start+query+date+location

driver.get(URL)

table = driver.find_element_by_xpath('//div[@class="line-chart"]/table/tbody')

print table.text
</code></pre>

<p>If I run the script, with an argument like "stackoverflow" I should be able to scrape this site: <a href="https://www.google.us/trends/explore?date=today%203-m&amp;geo=US&amp;q=stackoverflow" rel="nofollow">https://www.google.us/trends/explore?date=today%203-m&amp;geo=US&amp;q=stackoverflow</a></p>

<p>Apparently the xpath I have there is not working, the program is not printing anything, it's just plain blank. </p>

<p>I am basically in need on the values of the chart that appears on that website. And those values (and dates) are inside a table, here is a screenshot:</p>

<p><a href="http://i.stack.imgur.com/wmQkb.png" rel="nofollow"><img src="http://i.stack.imgur.com/wmQkb.png" alt="enter image description here"></a></p>

<p>Could you help me locate the correct xpath of the table to retrieve those values using selenium on python?</p>

<p>Thanks in advance!</p>

<p>you can use Xpath As Follow:</p>

<pre><code>//div[@class="line-chart"]/div/div[1]/div/div/table/tbody/tr
</code></pre>

<p>Here I will Refine my answer and make some changes in your code not it's work.</p>

<pre><code># NOTE: Download the chromedriver driver
# Then move exe file on C:\Python27\Scripts
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
import sys
from lxml.html import fromstring,tostring

driver = webdriver.Chrome()
driver.implicitly_wait(20)
'''
URL_start = "http://www.google.us/trends/explore?"
date = '&amp;date=today%203-m' # Last 90 days
location = "&amp;geo=US"
symbol = sys.argv[1]
query = 'q='+symbol
URL = URL_start+query+date+location
'''
driver.get("https://www.google.us/trends/explore?date=today%203-m&amp;geo=US&amp;q=stackoverflow")

table_trs = driver.find_elements_by_xpath('//div[@class="line-chart"]/div/div[1]/div/div/table/tbody/tr')

for tr in table_trs:
    #print tr.get_attribute("innerHTML").encode("UTF-8")

    td = tr.find_elements_by_xpath(".//td")
    if len(td)==2:
        print td[0].get_attribute("innerHTML").encode("UTF-8") +"\t"+td[1].get_attribute("innerHTML").encode("UTF-8")
</code></pre>

python selenium xpath web-scraping