Download a gzipped file, md5 checksum it, and then save extracted data if matches
<p>I'm currently attempting to download two files using Python, one a gzipped file, and the other, its checksum.</p>

<p>I would like to verify that the gzipped file's contents match the md5 checksum, and then I would like to save the contents to a target directory.</p>

<p>I found out how to download the files <a href="http://stackoverflow.com/questions/22676/how-do-i-download-a-file-over-http-using-python">here</a>, and I learned how to calculate the checksum <a href="http://stackoverflow.com/questions/16874598/how-do-i-calculate-the-md5-checksum-of-a-file-in-python">here</a>. I load the URLs from a JSON config file, and I learned how to parse JSON file values <a href="http://stackoverflow.com/questions/2835559/parsing-values-from-a-json-file-in-python">here</a>.</p>

<p>I put it all together into the following script, but I'm stuck attempting to store the verified contents of the gzipped file.</p>

<pre class="lang-python prettyprint-override"><code>import json
import gzip
import urllib
import hashlib

# Function for creating an md5 checksum of a file
def md5Gzip(fname):
    hash_md5 = hashlib.md5()

    with gzip.open(fname, 'rb') as f:
        # Make an iterable of the file and divide into 4096 byte chunks
        # The iteration ends when we hit an empty byte string (b"")
        for chunk in iter(lambda: f.read(4096), b""):
            # Update the MD5 hash with the chunk
            hash_md5.update(chunk)

    return hash_md5.hexdigest()

# Open the configuration file in the current directory
with open('./config.json') as configFile:
    data = json.load(configFile)

# Open the downloaded checksum file
with open(urllib.urlretrieve(data['checksumUrl'])[0]) as checksumFile:
    md5Checksum = checksumFile.read()

# Open the downloaded db file and get it's md5 checksum via gzip.open
fileMd5 = md5Gzip(urllib.urlretrieve(data['fileUrl'])[0])

if (fileMd5 == md5Checksum):
    print 'Downloaded Correct File'
    # save correct file
else:
    print 'Downloaded Incorrect File'
    # do some error handling
</code></pre>

<p>In your <code>md5Gzip</code>, return a <code>tuple</code> instead of just the hash.</p>

<pre><code>def md5Gzip(fname):
    hash_md5 = hashlib.md5()
    file_content = None

    with gzip.open(fname, 'rb') as f:
        # Make an iterable of the file and divide into 4096 byte chunks
        # The iteration ends when we hit an empty byte string (b"")
        for chunk in iter(lambda: f.read(4096), b""):
            # Update the MD5 hash with the chunk
            hash_md5.update(chunk)
        # get file content
        f.seek(0)
        file_content = f.read()

    return hash_md5.hexdigest(), file_content
</code></pre>

<p>Then, in your code:</p>

<pre><code>fileMd5, file_content = md5Gzip(urllib.urlretrieve(data['fileUrl'])[0])
</code></pre>

python io gzip md5 downloading