What is the mistake here?
<p>These are my codes but it seems to be correct,but it doesn't work,please help </p>

<pre><code>HEADER_XPATH = ['//h1[@class="story-body__h1"]//text()']    
AUTHOR_XPATH = ['//span[@class="byline__name"]//text()']   
PUBDATE_XPATH = ['//div/@data-datetime']  
WTAGS_XPATH = ['']   
CATEGORY_XPATH = ['//span[@rev="news|source""]//text()']    
TEXT = ['//div[@property="articleBody"]//p//text()']   
INTERLINKS = ['//div[@class="story-body__link"]//p//a/@href']  
DATE_FORMAT_STRING = '%Y-%m-%d'

class BBCSpider(Spider):
    name = "bbc"
    allowed_domains = ["bbc.com"]
    sitemap_urls = [
        'http://Www.bbc.com/news/sitemap/',
        'http://www.bbc.com/news/technology/',
        'http://www.bbc.com/news/science_and_environment/']

    def parse_page(self, response):
        items = []
        item = ContentItems()
        item['title'] = process_singular_item(self, response, HEADER_XPATH, single=True)
        item['resource'] = urlparse(response.url).hostname
        item['author'] = process_array_item(self, response, AUTHOR_XPATH, single=False)
        item['pubdate'] = process_date_item(self, response, PUBDATE_XPATH, DATE_FORMAT_STRING, single=True)
        item['tags'] = process_array_item(self, response, TAGS_XPATH, single=False)
        item['category'] = process_array_item(self, response, CATEGORY_XPATH, single=False)
        item['article_text'] = process_article_text(self, response, TEXT)
        item['external_links'] = process_external_links(self, response, INTERLINKS, single=False)
        item['link'] = response.url
        items.append(item)
        return items
</code></pre>

<p>Your spider is just badly structured and because of that it does nothing.<br>
The <code>scrapy.Spider</code> spider requires <code>start_urls</code> class attribute which should  contains list of urls that the spider will use to start the crawl, all of these urls will callback to class method <code>parse</code> which means it's required as well.</p>

<p>Your spider has <code>sitemap_urls</code> class attribute and it's not being used anywhere, also your spider has <code>parse_page</code> class method that is never used anywhere either.<br>
So in short your spider should look something like this:</p>

<pre><code>class BBCSpider(Spider):
    name = "bbc"
    allowed_domains = ["bbc.com"]
    start_urls = [
        'http://Www.bbc.com/news/sitemap/',
        'http://www.bbc.com/news/technology/',
        'http://www.bbc.com/news/science_and_environment/']

    def parse(self, response):
        # This is a page with all of the articles
        article_urls = # find article urls in the pages
        for url in article_urls:
            yield Request(url, self.parse_page)

    def parse_page(self, response):
        # This is an article page
        items = []
        item = ContentItems()
        # populate item
        return item
</code></pre>

python python-3.x scrapy web-crawler scrapy-spider