requests process hangs
<p>I'm using <code>requests</code> to get a URL, such as:</p>

<pre><code>while True:
    try:
        rv = requests.get(url, timeout=1)
        doSth(rv)
    except socket.timeout as e:
        print e
    except Exception as e:
        print e
</code></pre>

<p>After it runs for a while, it quits working. No exception or any error, just like it suspended. I then stop the process by typing Ctrl+C from the console. It shows that the process is waiting for data:</p>

<pre class="lang-none prettyprint-override"><code>  .............
   httplib_response = conn.getresponse(buffering=True) #httplib.py
   response.begin() #httplib.py
   version, status, reason = self._read_status() #httplib.py
   line = self.fp.readline(_MAXLINE + 1) #httplib.py
   data = self._sock.recv(self._rbufsize) #socket.py
KeyboardInterrupt
</code></pre>

<p>Why is this happening? Is there a solution?</p>

<p>It appears that the server you're sending your <code>request</code> to is throttling you - that is, it's sending <code>bytes</code> with less than 1 second between each package (thus not triggering your <code>timeout</code> parameter), but slow enough for it to appear to be stuck. </p>

<p>The only fix for this I can think of is to reduce the <code>timeout</code> parameter, unless you can fix this throttling issue with the Server provider. </p>

<p>Do keep in mind that you'll need to consider <code>latency</code> when setting the  <code>timeout</code> parameter, otherwise your connection will be dropped too quickly and might not work at all.</p>

python python-requests