scrapy starting a new project
<p>I have installed python 2.7.12 version on a Windows 7 system. I have also installed pywin32 and Visual C++. When I enter the command <code>pip --version</code> it does not generate any output, the cursor moves to the next line and blinks.</p>

<p>But when I use the command <code>python -m pip --version</code> the version of pip is displayed. Also to install scrapy I had to use the command <code>python -m pip install scrapy</code>. Scrapy got installed successfully. </p>

<p>I have set the path in the environment variables correctly - <code>C:\Python27;C:\Python27\Scripts;</code></p>

<p>When I had to start my new project in scrapy I used the command <code>scrapy startproject project_name</code>. Again, the cursor moved to the next line and blinked. No result is generated not even any error message.</p>

<p>When I tried again and again it created the folder in the directory with the respective files.</p>

<p>When I developed a code and tried to run the spider by the command <code>scrapy crawl name</code> again the same problem appeared - No response.</p>

<p>Now again I am not able to create a new project as the same issue is arising.</p>

<p>If someone could please suggest the possible reasons for the error and the solution for this.</p>

<p>It worked out 
when i used the command <code>python -m scrapy &lt;command&gt; &lt;arguments?</code> to follow the scrapy tutorial. however it was fine until i run the crawl command. when i use the <code>python -m scrapy.cmdline shell 'http://quotes.toscrape.com/page/1/'</code> command it shows error </p>

<pre><code>C:\Users\MinorMiracles\Desktop\tutorial&gt;python -m scrapy.cmdline crawl quotes
2016-10-19 10:26:15 [scrapy] INFO: Scrapy 1.2.0 started (bot: tutorial)
2016-10-19 10:26:15 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tu
torial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'ROBOTSTXT_OBEY': True,
 'BOT_NAME': 'tutorial'}
2016-10-19 10:26:16 [scrapy] INFO: Enabled extensions:
['scrapy.extensions.logstats.LogStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.corestats.CoreStats']
2016-10-19 10:26:17 [scrapy] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2016-10-19 10:26:17 [scrapy] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2016-10-19 10:26:17 [scrapy] INFO: Enabled item pipelines:
[]
2016-10-19 10:26:17 [scrapy] INFO: Spider opened
2016-10-19 10:26:17 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 i
tems (at 0 items/min)
2016-10-19 10:26:17 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023
2016-10-19 10:26:18 [scrapy] DEBUG: Crawled (404) &lt;GET http://quotes.toscrape.co
m/robots.txt&gt; (referer: None)
2016-10-19 10:26:18 [scrapy] DEBUG: Crawled (200) &lt;GET http://quotes.toscrape.co
m/page/1/&gt; (referer: None)
2016-10-19 10:26:18 [quotes] DEBUG: Saved file quotes-1.html
2016-10-19 10:26:18 [scrapy] DEBUG: Crawled (200) &lt;GET http://quotes.toscrape.co
m/page/2/&gt; (referer: None)
2016-10-19 10:26:19 [quotes] DEBUG: Saved file quotes-2.html
2016-10-19 10:26:19 [scrapy] INFO: Closing spider (finished)
2016-10-19 10:26:19 [scrapy] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 675,
 'downloader/request_count': 3,
 'downloader/request_method_count/GET': 3,
 'downloader/response_bytes': 5974,
 'downloader/response_count': 3,
 'downloader/response_status_count/200': 2,
 'downloader/response_status_count/404': 1,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2016, 10, 19, 4, 56, 19, 56000),
 'log_count/DEBUG': 6,
 'log_count/INFO': 7,
 'response_received_count': 3,
 'scheduler/dequeued': 2,
 'scheduler/dequeued/memory': 2,
 'scheduler/enqueued': 2,
 'scheduler/enqueued/memory': 2,
 'start_time': datetime.datetime(2016, 10, 19, 4, 56, 17, 649000)}
2016-10-19 10:26:19 [scrapy] INFO: Spider closed (finished)

C:\Users\MinorMiracles\Desktop\tutorial&gt;python -m scrapy.cmdline shell 'http://q
uotes.toscrape.com/page/1/'
2016-10-19 11:11:40 [scrapy] INFO: Scrapy 1.2.0 started (bot: tutorial)
2016-10-19 11:11:40 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tu
torial.spiders', 'ROBOTSTXT_OBEY': True, 'DUPEFILTER_CLASS': 'scrapy.dupefilters
.BaseDupeFilter', 'SPIDER_MODULES': ['tutorial.spiders'], 'BOT_NAME': 'tutorial'
, 'LOGSTATS_INTERVAL': 0}
2016-10-19 11:11:40 [scrapy] INFO: Enabled extensions:
['scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.corestats.CoreStats']
2016-10-19 11:11:40 [scrapy] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2016-10-19 11:11:40 [scrapy] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2016-10-19 11:11:40 [scrapy] INFO: Enabled item pipelines:
[]
2016-10-19 11:11:40 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023
2016-10-19 11:11:40 [scrapy] INFO: Spider opened
2016-10-19 11:11:42 [scrapy] DEBUG: Retrying &lt;GET http://'http:/robots.txt&gt; (fai
led 1 times): DNS lookup failed: address "'http:" not found: [Errno 11004] getad
drinfo failed.
2016-10-19 11:11:45 [scrapy] DEBUG: Retrying &lt;GET http://'http:/robots.txt&gt; (fai
led 2 times): DNS lookup failed: address "'http:" not found: [Errno 11004] getad
drinfo failed.
2016-10-19 11:11:47 [scrapy] DEBUG: Gave up retrying &lt;GET http://'http:/robots.t
xt&gt; (failed 3 times): DNS lookup failed: address "'http:" not found: [Errno 1100
4] getaddrinfo failed.
2016-10-19 11:11:47 [scrapy] ERROR: Error downloading &lt;GET http://'http:/robots.
txt&gt;: DNS lookup failed: address "'http:" not found: [Errno 11004] getaddrinfo f
ailed.
DNSLookupError: DNS lookup failed: address "'http:" not found: [Errno 11004] get
addrinfo failed.
2016-10-19 11:11:49 [scrapy] DEBUG: Retrying &lt;GET http://'http://quotes.toscrape
.com/page/1/'&gt; (failed 1 times): DNS lookup failed: address "'http:" not found:
[Errno 11004] getaddrinfo failed.
2016-10-19 11:11:51 [scrapy] DEBUG: Retrying &lt;GET http://'http://quotes.toscrape
.com/page/1/'&gt; (failed 2 times): DNS lookup failed: address "'http:" not found:
[Errno 11004] getaddrinfo failed.
2016-10-19 11:11:54 [scrapy] DEBUG: Gave up retrying &lt;GET http://'http://quotes.
toscrape.com/page/1/'&gt; (failed 3 times): DNS lookup failed: address "'http:" not
 found: [Errno 11004] getaddrinfo failed.
Traceback (most recent call last):
  File "C:\Python27\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 161, in &lt;module&gt;
    execute()
  File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 142, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 88, in _run_print
_help
    func(*a, **kw)
  File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 149, in _run_comm
and
    cmd.run(args, opts)
  File "C:\Python27\lib\site-packages\scrapy\commands\shell.py", line 71, in run

    shell.start(url=url)
  File "C:\Python27\lib\site-packages\scrapy\shell.py", line 47, in start
    self.fetch(url, spider)
  File "C:\Python27\lib\site-packages\scrapy\shell.py", line 112, in fetch
    reactor, self._schedule, request, spider)
  File "C:\Python27\lib\site-packages\twisted\internet\threads.py", line 122, in
 blockingCallFromThread
    result.raiseException()
  File "&lt;string&gt;", line 2, in raiseException
twisted.internet.error.DNSLookupError: DNS lookup failed: address "'http:" not f
ound: [Errno 11004] getaddrinfo failed.
</code></pre>

<p>can anyone tell me what is wrong</p>

<p>Using the alternative command <code>python -m scrapy.cmdline &lt;command&gt; &lt;arguments&gt;</code> (e.g. python -m scrapy.cmdline version -v) worked</p>

<p>thanks Paul</p>

python python-2.7 web-scraping scrapy scrapy-spider