Scrapy Pipeline not starting
<p>I'm having problems with Scrapy pipelines.
EnricherPipeline is never starting. I put a debugger in the fist line of process_item and it never gets control.
JsonPipeline does start, but the first argument it receives is of type <code>generator object process_item</code> and not the MatchItem instance it should receive (when I disable the EnricherPipeline, JsonPipeline works as expected.</p>

<pre><code>class MatchSpider(CrawlSpider):

    def parse(self, response):
        browser = Browser(browser='Chrome')
        browser.get(response.url)
        browser.find_element_by_xpath('//a[contains(text(), "{l}") and @title="{c}"]'.format(l=self.league, c=self.country)).click()
        browser.find_element_by_xpath('//select[@id="seasons"]/option[text()="{s}"]'.format(s=self.season.replace('-', '/'))).click()
        browser.find_element_by_xpath('//a[contains(text(), "Fixture")]').click()
        page_matches = browser.find_elements_by_xpath('//*[contains(@class, "result-1 rc")]')
        matches.extend([m.get_attribute('href') for m in page_matches]
        for m in matches[:1]:
            yield Request(m, callback=self.process_match, dont_filter=True)

    def process_match(self, response):
        match_item = MatchItem()
        match_item['url'] = response.url
        match_item['project'] = self.settings.get('BOT_NAME')
        match_item['spider'] = self.name
        match_item['server'] = socket.gethostname()
        match_item['date'] = datetime.datetime.now()
        return match_item

class EnricherPipeline:
    def process_item(self, item, spider):
        self.match = defaultdict(dict)
        self.match['date'] = item['match']['startTime']
        self.match['referee'] = item['match']['refereeName']
        self.match['stadium'] = item['match']['venueName']
        self.match['exp_mins'] = item['match']['expandedMinutes']
        yield self.match


class JsonPipeline:

    def process_item(self, item, scraper):
        output_dir = 'data/matches/{league}/{season}'.format(league=scraper.league, season=scraper.season)
        if not os.path.exists(output_dir):
            os.makedirs(output_dir)
        file_name = "-".join([str(datetime.strptime(item['date'], '%Y-%m-%dT%H:%M:%S').date()),
                              item['home']['name'], item['away']['name']]) + '.json'
        item_path = os.sep.join((output_dir, file_name))
        with open(item_path, 'w') as f:
            f.write(json.dumps(item))



ITEM_PIPELINES = {
    'scrapers.whoscored.whoscored.pipelines.EnricherPipeline': 300,
    'scrapers.whoscored.whoscored.pipelines.JsonPipeline': 800,
}
</code></pre>

<p>Ok, so the problem was that EnricherPipeline was yielding and not returning a result. After that it worked as expected, although I still don't understand why a debugger is not working in that first pipeline.</p>

python scrapy