How to Catch a 503 Error and retry request
<p>I'm using <code>grequests</code> to make about 10,000 calls, but some of these calls return as <code>503</code>. This problem goes away if I don't queue all 10,000 calls at once. Breaking it into groups of 1000 seems to do the trick. However I was wondering if there's a way to catch this <code>503</code> error and just retry the <code>request</code>.</p>

<p>This is how I'm calling and combining the threads:</p>

<pre><code>import grequests
rs = (grequests.get(u, headers=header) for u in urls)
response = grequests.map(rs)
</code></pre>

<p>I know this is really vague, but I don't even know if this is possible using <code>grequests</code>. </p>

<p>I naivley tried </p>

<pre><code>import grequests
rs = (grequests.get(u, headers=header) for u in urls)
time.sleep(1)
response = grequests.map(rs)
</code></pre>

<p>But this does nothing to slow it down. </p>

<p>Maybe you can try using event hooks to catch the error and re-launch the requests <a href="http://docs.python-requests.org/en/master/user/advanced/#event-hooks" rel="nofollow">http://docs.python-requests.org/en/master/user/advanced/#event-hooks</a></p>

<pre><code> import grequests

 def response_handler(response):
     if response.status_code == '503':
         print('error.503')

 rs = (grequests.get(u, headers=header, hooks = dict('response' : response_handler)) for u in urls)
 response = grequests.map(rs)`
</code></pre>

python python-requests grequests