Retrieveing files form URL in Python returns blank
<p>I'm working on a chess related project for which I have to download a very large quantity of files from <a href="http://chesstempo.com" rel="nofollow">ChessTempo</a>. </p>

<p>When running the following code:</p>

<pre><code>import urllib.request

url = "http://chesstempo.com/requests/download_game_pgn.php?gameids="

for i in range (3,500):
    urllib.request.urlretrieve(url + str(i),'Games/Game ' + str(i) + ".pgn")
    print("Downloaded file nÂº " + str(i))
</code></pre>

<p>I get the expected list of 500~ files but they are all blank except the second and third files, which have the correct data in them.</p>

<p>When I open the URLs by hand, it all works perfectly. What am I missing?</p>

<p>In fact, I can only download files 2 &amp; 3, all others are empty...</p>

<p>Were you logged in while accessing those files "manually"? (Which I assume to be using a web browser).</p>

<p>If so, FYI an http request does not only consist of the URL, lots of other information is transfered. So if you are not getting the same information, you are almost certainly not making the same request.</p>

<p>In chrome you can see the requests you make within a page.</p>

<p>From <em>Developer Tools</em> go to <em>Network</em> > <em>Select a name form the list</em> > <em>Request Headers</em> (<a href="https://i.stack.imgur.com/GORS4.png" rel="nofollow">See picture</a>)</p>

<p>The most probable thing that you may be looking for are the <em>cookies</em></p>

<p>Hope it helps.</p>

python urllib