How to send a repeat request in urllib2.urlopen in Python if the first call is just stuck?
<p>I am making a call to a URL in Python using urllib2.urlopen in a while(True) loop</p>

<p>My URL keeps changing every time (as there is a change in a particular parameter of the URL every time).</p>

<p>My code look as as follows:</p>

<pre><code>def get_url(url):
    '''Get json page data using a specified API url'''
    response = urlopen(url)
    data = str(response.read().decode('utf-8'))
    page = json.loads(data)
    return page
</code></pre>

<p>I am calling the above method from the main function by changing the url every time I make the call.</p>

<p>What I observe is that after few calls to the function, suddenly (I don;t know why), the code gets stuck at the statement</p>

<pre><code>response = urlopen(url)
</code></pre>

<p>and it just waits and waits...</p>

<p>How do I best handle this situation?</p>

<p>I want to make sure that if it does not respond within say 10 seconds, I make the same call again.</p>

<p>I read about </p>

<pre><code>response = urlopen(url, timeout=10)
</code></pre>

<p>but then what about the repeated call if this fails?</p>

<p>Depending on how many retries you want to attempt, use a try/catch inside a loop:</p>

<pre><code>while True:
    try:
        response = urlopen(url, timeout=10)
        break
    except:
        # do something with the error
        pass
# do something with response
data = str(response.read().decode('utf-8'))
...  
</code></pre>

<p>This will silence all exceptions, which may not be ideal (more on that here: <a href="http://stackoverflow.com/questions/2712524/handling-urllib2s-timeout-python">Handling urllib2&#39;s timeout? - Python</a>)</p>

python http urllib2