How to download large files in Python 2
<p>I'm trying to download large files (approx. 1GB) with mechanize module, but I have been unsuccessful. I've been searching for similar threads, but I have found only those, where the files are publicly accessible and no login is required to obtain a file. But this is not my case as the file is located in the private section and I need to login before the download. Here is what I've done so far.</p>

<pre><code>import mechanize

g_form_id = ""

def is_form_found(form1):
    return "id" in form1.attrs and form1.attrs['id'] == g_form_id

def select_form_with_id_using_br(br1, id1):
    global g_form_id
    g_form_id = id1
    try:
        br1.select_form(predicate=is_form_found)
    except mechanize.FormNotFoundError:
        print "form not found, id: " + g_form_id
        exit()

url_to_login = "https://example.com/"
url_to_file = "https://example.com/download/files/filename=fname.exe"
local_filename = "fname.exe"

br = mechanize.Browser()
br.set_handle_robots(False)   # ignore robots
br.set_handle_refresh(False)  # can sometimes hang without this
br.addheaders =  [('User-agent', 'Firefox')]

response = br.open(url_to_login)
# Find login form
select_form_with_id_using_br(br, 'login-form')
# Fill in data
br.form['email'] = 'email@domain.com'
br.form['password'] = 'password'
br.set_all_readonly(False)    # allow everything to be written to
br.submit()

# Try to download file
br.retrieve(url_to_file, local_filename)
</code></pre>

<p>But I'm getting an error when 512MB is downloaded:</p>

<pre><code>Traceback (most recent call last):
  File "dl.py", line 34, in &lt;module&gt;
    br.retrieve(br.retrieve(url_to_file, local_filename)
  File "C:\Python27\lib\site-packages\mechanize\_opener.py", line 277, in retrieve
    block = fp.read(bs)
  File "C:\Python27\lib\site-packages\mechanize\_response.py", line 199, in read
    self.__cache.write(data)
MemoryError: out of memory
</code></pre>

<p>Do you have any ideas how to solve this?
Thanks</p>

<p>Try downloading/writing it by chunks. Seems like file takes all your memory.</p>

<p>You should specify Range header for your request if server supports it.</p>

<p><a href="https://en.wikipedia.org/wiki/List_of_HTTP_header_fields" rel="nofollow">https://en.wikipedia.org/wiki/List_of_HTTP_header_fields</a></p>

python download mechanize