Extracting text within frame tree in html
<p>I would like to scrape a frame within a website (<a href="https://www.harris.com/careers/jobs" rel="nofollow">https://www.harris.com/careers/jobs</a>). THe Xpath for the location for the first listed job position is </p>

<pre><code>/html/body/center/table[2]/tbody/tr/td/form/table[3]/tbody/tr[3]/td/table/tbody/tr[3]/td[4]/span
</code></pre>

<p>I am attempting to extract the text within the span using lxml library in Python. My code is currently as follows</p>

<pre><code>from lxml import html
import requests
page = requests.get('https://www.harris.com/careers/jobs')
tree = html.fromstring(page.content)

location = tree.xpath('/html/body/center/table[2]/tbody/tr/td/form/table[3]/tbody/tr[3]/td/table/tbody/tr[3]/td[4]/span/text()')
</code></pre>

<p>Unfortunately the command </p>

<pre><code>print(test)
</code></pre>

<p>yields the following</p>

<pre><code>[]
</code></pre>

<p>I'm quite sure theres something wrong with the Xpath and that it can be improved to extract the text i need. Any help would be appreciated. Thanks.</p>

<p>Here I will give working code for that:</p>

<pre><code>from selenium.webdriver.common.keys import Keys
from selenium import webdriver
import time

driver=webdriver.Chrome('./chromedriver.exe')
try:
    driver.get("https://www.harris.com/careers/jobs")
    driver.switch_to.frame("frmJobs");
    time.sleep(5)
    #s = driver.find_element_by_id("searchbuttonBtn__a")
    s = driver.find_element_by_xpath("//input[@class='submitbutton']")
    driver.execute_script("return arguments[0].scrollIntoView();",s)
    print s.get_attribute("value")
    s.send_keys("\n")
    time.sleep(10)
    for a in driver.find_elements_by_xpath("//td[@class='listheadingbackground']/table/tbody/tr/td[2]/span/a"):
        print a.get_attribute("href")
except Exception as e:
    print e
driver.quit()
</code></pre>

python xpath web-scraping lxml python-3.5