Read a complete data file and round numbers to 2 decimal places and save it with the same format
<p>I am trying to learn python and I have the intention to make the a very big data file smaller and later do some statistical Analysis with R. I need to read the data file (see below):</p>

<pre><code>SCALAR
ND    3
ST  0
TS        10.00
0.0000
0.0000
0.0000
SCALAR
ND    3
ST  0
TS      3600.47
255.1744
255.0201
255.2748
SCALAR
ND    3
ST  0
TS      7200.42
255.5984
255.4946
255.7014
</code></pre>

<p>and find the numbers and round it in two digits after decimal, svae the maximum number with the namber in front of TS. At the end save the data file with the same format like following:</p>

<pre><code>SCALAR
ND    3
ST  0
TS        10.00
0.00
0.00
0.00
SCALAR
ND    3
ST  0
TS      3600.47
255.17
255.02
255.27
SCALAR
ND    3
ST  0
TS**MAX**      7200.42
255.60
255.49
255.70
</code></pre>

<p>I have written a code like this:</p>

<pre><code>import numpy as np
import matplotlib.pyplot as plt
import pickle

# Open file
f = open('data.txt', 'r')
thefile = open('output.txt', 'wb')
# Read and ignore header lines
header1 = f.readline()
header2 = f.readline()
header3 = f.readline()
header4 = f.readline()
data = []
for line in f:
    line = line.strip()
    columns = line.split()
    source = {}
    source['WSP'] = columns[0]
    #source['timestep'] = float(columns[1])
    source['timestep'] = columns[1]
    data.append(source)

f.close()
</code></pre>

<p>but the number in front of TS cannot be read. I wanted to round the numbers but the float which I used does not work. After that I wanted to put it in a loop Any suggestion, Do I write the code in a good way? I will be very thankfull for the help.</p>

<p>The first part can be done easily if it can be assumed that the floats that require rounding occur only on lines by themselves. That excludes lines that are prefixed with alpha chars, e.g. <code>TS      3600.47</code>.</p>

<pre><code>from __future__ import print_function

with open('data.txt') as f, open('output.txt', 'w') as outfile:
    for line in (l.rstrip() for l in f):
        try:
            print('{:.2f}'.format(float(line)), file=outfile)
        except ValueError:
            print(line, file=outfile)
</code></pre>

<p>The second part, however, requires that the file be buffered in its entirety because it is not known where the maximum value for <code>TS</code> will be -  it could be at the start of the file, at the end, or anywhere in between. Here's some code to do that:</p>

<pre><code>from __future__ import print_function

with open('data.txt') as f, open('output.txt', 'w') as outfile:
    lines = []
    max_ts = 0
    max_ts_idx = None

    for i, line in enumerate(l.rstrip() for l in f):
        try:
            lines.append('{:.2f}'.format(float(line)))
        except ValueError:
            if line.startswith('TS'):
                new_ts = float(line.split()[-1])
                if new_ts &gt; max_ts:
                    max_ts = new_ts
                    max_ts_idx = i
            lines.append(line)

    for i, line in enumerate(lines):
        if i == max_ts_idx:
            line = line.replace('TS', 'TS**MAX**')
        print(line, file=outfile)
</code></pre>

<p>It's basically the same as the print only version above, however, the lines are now accumulated into the list <code>lines</code>. The maximum value for "TS" lines is kept in <code>max_ts</code> and the corresponding line number of that "TS" line in <code>max_ts_idx</code>. Finally the <code>lines</code> list is iterated over and the lines are written to the file. If the line contains the maximum value for "TS" (as determined by <code>max_ts_idx</code>) that line is decorated with <code>**MAX**</code>.</p>

python statistics string-formatting data-type-conversion