Why getting Memory Error? Python
<p>I have a 5gb text file and i am trying to read it line by line. 
My file is in format-: Reviewerid&lt;\t>pid&lt;\t>date&lt;\t>title&lt;\t>body&lt;\n>
This is my code </p>

<pre><code>o = open('mproducts.txt','w')
with open('reviewsNew.txt','rb') as f1:
    for line in f1:
        line = line.strip()
        line2 = line.split('\t')
        o.write(str(line))
        o.write("\n")
</code></pre>

<p>But i get Memory error when i try to run it. I have an 8gb ram and 1Tb space then why am i getting this error? I tried to read it in blocks but then also i get that error.</p>

<pre><code>MemoryError 
</code></pre>

<p><strong>Update:</strong></p>

<p>Installing 64 bit Python solves the issue.</p>

<p>OP was using 32 bit Python that's why getting into memory limitation.</p>

<hr>

<p>Reading whole comments I think this can help you.</p>

<ul>
<li>You can't read file in chunk (as 1024) since you want to process data.</li>
<li>Instead, read file in chunk of lines i.e N lines at a time.</li>
<li>You can use <a href="http://stackoverflow.com/questions/231767/what-does-the-yield-keyword-do">yield</a> keyword and <a href="https://pymotw.com/2/itertools/" rel="nofollow">itertools</a> in Python to achieve above. </li>
</ul>

<p><strong><em>Summary</strong> : Get N lines at time, process it and then write it.</em></p>

<p><strong>Sample Code :</strong></p>

<pre><code>from itertools import islice
#You can change num_of_lines
def get_lines(file_handle,num_of_lines = 10):
    while True:
        next_n_lines = list(islice(file_handle, num_of_lines))
        if not next_n_lines:
            break
        yield next_n_lines


o = open('mproducts.txt','w')

with open('reviewsNew.txt','r') as f1:
    for data_lines in get_lines(f1):
        for line in data_lines:
            line = line.strip()
            line2 = line.split('\t')
            o.write(str(line))
            o.write("\n")
o.close()
</code></pre>

python