write into file without reading all the file
<p>I have a template of a file (html) with the header and footer. I try to insert text into right after <code>&lt;trbody&gt;</code>.
The way i'm doing it right now is with <code>fileinput.input()</code> </p>

<pre><code>def write_to_html(self,path):

for line in fileinput.input(path, inplace=1):
    line = re.sub(r'CURRENT_APPLICATION', obj, line)
    line = re.sub(r'IN_PROGRESS', time.strftime("%Y-%m-%d %H:%M:%S"), line)
    line = re.sub(r'CURRENT_VERSION', svers, line)
    print line,  # preserve old content
    if "&lt;tbody&gt;" in line:
        print ("&lt;tr&gt;")  
        ###PRINT MY STUFFS
        print ("&lt;/tr&gt;")
</code></pre>

<p>I call this for each Table-line I have to add in my html table. but I have around 5k table-lines to add (each line is about 30 lines of hmtl code). It starts fast, but each line takes more and more times to be added. It's because it has to write the file all over again for each line right ?</p>

<p>Is there a way to speed up the process?</p>

<p>EDIT thanks for the responses :
I like the idee of creating my big string, and the just go through the file just once.
I'll have to change some stuff because right now because the function I showed is in a Classe. and in my main programe, I just iterate on a folder containing .json.</p>

<pre><code> for json in jsonfolder :
     Object_a = CLASS-A(json) #unserialization
     Object_a.write_to_html()   (the function i showed)
</code></pre>

<p>I should turn that into :</p>

<pre><code>block_of_lines=''
for json in jsonfolder :
    Object_a = CLASS-A(json) #unserialization
    block_of_line += Object_a.to_html_sting()
Create_html(block_of_line)
</code></pre>

<p>Would that be faster ?</p>

<p>Re-reading the question a couple more times, the following thought occurs.
Could you split the writing into 3 blocks - one for the header, one for the table lines and another for the footer. It does rather seem to depend on what those three substitution lines are doing, but if I'm right, they can only update lines the first time the template is used, ie. while acting on the first json file, and then remain unchanged for the others.</p>

<pre><code> file_footer = CLASS-A.write_html_header(path)
 for json in jsonfolder :
     Object_a = CLASS-A(json) #unserialization
     Object_a.write_to_html(path)   #use the part of the function
                                # that just handles the json file here
 CLASS-A.write_html_footer(path, footer)
</code></pre>

<p>Then in your class, define the two new functions to write the header and footer as static methods (which means they can be used from the class rather than just on an instance)
i.e. (using a copy from your own code)</p>

<pre><code>@staticmethod
def write_html_header(path):
    footer = []
    save_for_later = false
    for line in fileinput.input(path, inplace=1):
        line = re.sub(r'CURRENT_APPLICATION', obj, line)
        line = re.sub(r'IN_PROGRESS', time.strftime("%Y-%m-%d %H:%M:%S"), line)
        line = re.sub(r'CURRENT_VERSION', svers, line)
        # this blocks prints the header, and saves the
        # footer from your template.
        if save_for_later:
            footer.append(line)
        else:
            print line,  # preserve old content
        if "&lt;tbody&gt;" in line:
            save_for_later = true
    return footer
</code></pre>

<p>I do wonder why you're editing 'inplace' doesn't that mean the template get's overwritten, and thus it's less of a template and more of a single use form. Normally when I use a template, I read in from the template, and write out to a new file an edited version of the template. Thus the template can be re-used time and time again.</p>

<p>For the footer section, open your file in append mode, and then write the lines in the footer array created by the call to the header writing function.</p>

<p>I do think not editing the template in place would be of benefit to you. then you'd just need to :</p>

<pre><code>open the template (read only)
open the new_file (in new, write mode)
write the header into new_file
loop over json files
    append table content into new_file
append the footer into new_file
</code></pre>

<p>That way you're never re-reading the bits of the file you created while looping over the json files. Nor are you trying to store the whole file in memory if that is a concern.</p>

python python-2.7