Python performance on reading files with extremely long lines
<p>I've got a file with around 6MB of data. All of the data are written in a single line. Why is the following command taking more than 15 minutes to finish? Is it normal?</p>

<pre><code>infile = open('file.txt')
outfile = open('out.txt', 'w')
for line in infile.readlines():
    outfile.write(line);
</code></pre>

<p><strong>Details:</strong></p>

<p>I'm using Python 2.7.</p>

<p>Output from <strong>wc</strong>:  </p>

<ul>
<li>newline count: 2</li>
<li>word count: 3475246 </li>
<li>byte count: 6951140 </li>
</ul>

<p><strong>Evaluation 1:</strong></p>

<p><em>Reference</em> code using in file (suggestted by Ahsanul Haque and Daewon Lee):</p>

<pre><code>for line in infile:
    output.write(line):
</code></pre>

<p><em>Time: 959.487 secs.</em></p>

<p>try reading file in chunks</p>

<pre><code>infile = open('file.txt')
outfile = open('out.txt', 'w')
while True:
    text = infile.read(100): # or any other size
    if not text:
        break
    outfile.write(line);
</code></pre>

python performance file