parse html tables with lxml
<p>I have been trying to parse the table contents from <a href="https://nseindia.com/products/content/derivatives/equities/fo_underlying_home.htm" rel="nofollow">here</a>
i have tried a couple of alternatives, like </p>

<pre><code>xpath('//table//tr/td//text()') 
xpath('//div[@id="replacetext"]/table/tbody//tr/td/a//text()')
</code></pre>

<p>here is my last code:</p>

<pre><code>import requests, lxml.html
url ='https://nseindia.com/products/content/derivatives/equities/fo_underlying_home.htm'
url = requests.get(url)
html = lxml.html.fromstring(url.content)
packages = html.xpath('//div[@id="replacetext"]/table/tbody//tr/td/a//text()') # get the text inside all "&lt;tr&gt;&lt;td&gt;&lt;a ...&gt;text&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;"
</code></pre>

<p>however none of the alternatives seems to be working. In the past, i have scraped data with similar code (although not from this url!). Any guidance will be really helpful.</p>

<p>I tried you code. The problem is not caused by <code>lxml</code>. It is caused by how you load the webpage.</p>

<p>I know that you use the <code>requests</code> to get the content of webpage, however,  the content you get from <code>requests</code> may be different from the content you see in the browser.</p>

<p>In this page, '<a href="https://nseindia.com/products/content/derivatives/equities/fo_underlying_home.htm" rel="nofollow">https://nseindia.com/products/content/derivatives/equities/fo_underlying_home.htm</a>', print the content of <code>request.get</code>, you will find that the source code of this page contains no table!!! The table is loaded by <code>ajax</code> query.</p>

<p>So find a way to load the 'really' page you want, the you can use 'lxml`.</p>

<p>By the way, in web scraping, there are also something you need to mention, for example, <code>request headers</code>. It's a good practice to set your request headers when you do the http request. Some sites may block you, if you do not provide a reasonable <code>User-Agent</code> in the header. Though there is nothing to do with your current problem.</p>

<p>Thanks.</p>

python parsing lxml