Scrapy - Use proxy middleware but disable proxy for specific requests
<p>I want to use proxy middleware in my Scrapy but not every request needs a proxy. I don't want to abuse the proxy usage and make the proxy prone to get banned.</p>

<p>Is there a way for me to disable proxy in some requests when the proxy middleware is turned on?</p>

<p>It's in the <a href="http://scrapy.readthedocs.io/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpproxy" rel="nofollow">docs</a>.  </p>

<p>You can set the meta key <code>proxy</code> per-request, to a value like <code>http://some_proxy_server:port</code>.</p>

python web-scraping scrapy screen-scraping scrapy-spider