Can I pass a file to Popen and still have it run asynchronously
<p>I have a Django based server and I'm calling a script which does a bunch of work. This needs to be asynchronous so I'm using Popen. However for debugging I want to redirect stdout and stderr from PIPE to a file. Will this affect the asynchronous performance?</p>

<p>How should I make sure the file opens and closes properly when the python script itself is already done (and has made a return call.)</p>

<p><code>Popen</code> runs asynchronously by default.</p>

<p>The <code>Popen</code> object can be stored and queried later.</p>

<pre><code>p = subproccess.Popen(executable)

# continue with other work

p.poll() # returns None if p is still running, returncode otherwise
p.terminate() # closes the process forcefully

p.wait() # freezes the execution until process finishes - makes it synchronous
</code></pre>

<p>All that is needed is to store <code>p</code> somewhere until its status has to be checked, e.g. in a global list of started processes, if there is no better place.</p>

<p><strong>UPDATE</strong></p>

<p>The problem seems to be how to open a file, give it to <code>Popen</code> and then close the file when the process is finished, without keeping any references.</p>

<p>This can be simply done with a thread:</p>

<pre><code>def run_process(executable, filename):
    with open(filename, 'w') as f:
        subprocess.Popen(executable, stdout=f).wait()

Thread(target=run_process, args=(executable, filename)).start()
</code></pre>

<p>Run it and forget about it. <code>run_process</code> will freeze until the process finishes and then it will close the file, but that all happens in a different thread.</p>

<p><strong>Note:</strong> you may or may not care about what happens to the thread or to the process if they are not finished when the django process finishes.</p>

<p>For handling that, you might create a more complex thread which you would remember and stop when before django exits; it may also close the process or wait for it to finish...</p>

python asynchronous subprocess