How to extract all href content from a page using scrapy
<p>I am trying to crawl <a href="https://www.goodreads.com/list/show/19793.I_Marked_My_Calendar_For_This_Book_s_Release" rel="nofollow">https://www.goodreads.com/list/show/19793.I_Marked_My_Calendar_For_This_Book_s_Release</a>.</p>

<p>I want to get all links from a given website using Scrapy</p>

<p>I am trying to this way -</p>

<pre><code>import scrapy
import unidecode
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
from lxml import html


class ElementSpider(scrapy.Spider):
    name = 'linkdata'

    start_urls = ["https://www.goodreads.com/list/show/19793.I_Marked_My_Calendar_For_This_Book_s_Release",]


    def parse(self, response):

        links = response.xpath('//div[@id="all_votes"]/table[@class="tableList js-dataTooltip"]/div[@class="js-tooltipTrigger tooltipTrigger"]/a/@href').extract()
        print links
</code></pre>

<p>But i am getting nothing in output.</p>

<p>I think your xpath is worng. Try this-</p>

<pre><code>for href in response.xpath('//div[@id="all_votes"]/table[@class="tableList js-dataTooltip"]/tr/td[2]/div[@class="js-tooltipTrigger tooltipTrigger"]/a/@href'):       
            full_url = response.urljoin(href.extract())
            print full_url
</code></pre>

<p>Hope it helps :)</p>

<p>Good luck...</p>

python scrapy