How can I pause a long running loop?
<p>I have a question about pausing / resuming a long running program. I'll use Python in this example, but it could be for any programming language really.</p>

<p>Let's say I want to sum up all the numbers to a billion for example</p>

<pre><code>results = []

for i in range(1000000000):
    results.append(i + i*2)

with open('./sum.txt', 'w') as outfile:
    for r in results:
        output.write(r+'\n')
outfile.close()
</code></pre>

<p>That's a very simple program and in the end I would like to write the values out to a file.</p>

<p>Let's say I start this script and it's taking hours and I want to shut down my computer. Now I could wait but what are some ways that I could stop this process while keeping the data in tact?</p>

<p>I've though about opening the file before the loop and inside the loop, instead of appending to the results list, I could just write [append] to the opened file but that seems like still if I quit the program, the file never closes and the data could get corrupted. Even if I just write to the file in the loop then directly close the file after the write operation, couldn't that still cause data corruption or something?</p>

<p>What are some ways that I could tackle a problem like this. Where I'd like to be able to stop a program and later, restart the program and it could continue from where it left off?</p>

<p>If you have a long running script that does most of the work on the inner part of a loop.</p>

<blockquote>
  <p>@MisterMiyagi 
  Can you please specify what your actual question is?
  Creating safe-points inside the loop is the way to go if you want to
  preserve the loop's state. Whether file I/O is safe against corruption
  doesn't depend on your application so much as operating/file system.
  In Linux, it's safe to write to a temporary file, then rename it;
  files are also closed automatically when a program exits.</p>
</blockquote>

<p>I was more interested in a safe place inside the loop. What are some recommended ways to do that in a program that's structured like the one above.</p>

<h3>Create a condition for the loop to either continue or hold</h3>

<p>A possible solution would be in the script, periodically checking for a "trigger" file to exist. In the example below, the loop checks once in hundred cycles, but you could make it 500 for example.</p>

<p><strong><em>If</em></strong> and as long as the file exists, the loop will hold, checking again every two seconds. <br>
While this works fine, you will have to accept the fact that it slightly slows down the loop, depending on <code>n</code>; larger n will decrease the effect. <em>ANY</em> solution inside the loop will however have at least <em>some</em> effect on the loop.</p>

<p>Although this is on <code>python</code>, the concept should be possible in any language, and on any os.</p>

<p>The <code>print</code> functions are only to show that it works of course:)</p>

<h3>The edited loop:</h3>

<pre><code>import os
import time

results = []
n = 100

for i in range(1000000000):
    results.append(i + i*2)
    print(i)
    if i%n == 0:
        while os.path.exists("test123"):
            time.sleep(2)
            print("wait")   
        else:
            print("go on")

with open('./sum.txt', 'w') as outfile:
    for r in results:
        output.write(r+'\n')
outfile.close()
</code></pre>

<p>The result, if I create the trigger file and remove it again:</p>

<pre><code>4096
4097
4098
4099
wait
wait
wait
wait
wait
wait
go on
4100
4101
4102
</code></pre>

python loops for-loop long-running-processes pause