Cannot successfully redirect stdout from Popen to temp file
<p>The title says it all, really. Currently running Python 2.7.8, using <code>subprocess32</code>. Minimum reproducing snippet, copied directly from the REPL:</p>

<pre><code>Python 2.7.8 (default, Aug 14 2014, 13:26:38)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; import tempfile
&gt;&gt;&gt; import subprocess32 as subp
&gt;&gt;&gt; tfil = tempfile.TemporaryFile()
&gt;&gt;&gt; tfil
&lt;open file '&lt;fdopen&gt;', mode 'w+b' at 0x7f79dcf158a0&gt;
&gt;&gt;&gt; proc = subp.Popen(['ls', '-al'], stdout=tfil, stderr=subp.STDOUT)
&gt;&gt;&gt; proc
&lt;subprocess32.Popen object at 0x7f79dcf04a50&gt;
&gt;&gt;&gt; proc.wait()
0
&gt;&gt;&gt; tfil.read()
''
&gt;&gt;&gt;
</code></pre>

<p>Changes to the code that made no difference in the result, in any combination:</p>

<ul>
<li>Using <code>subprocess</code> instead of <code>subprocess32</code></li>
<li>Using <code>NamedTemporaryFile</code> instead of <code>TemporaryFile</code></li>
<li>Setting <code>bufsize=0</code> on either or both of <code>tfil</code> and <code>proc</code></li>
<li>If I try <code>subp.Popen(['ls', '-al'], stdout=sys.stdout, stderr=subp.STDOUT)</code>, the output redirects to the terminal just fine - but I need to capture it somewhere else, I can't just mix it together with actual terminal output. Redirecting <code>sys.stdout</code> to the temp file, and then setting it back after the process is done, doesn't work either.</li>
</ul>

<p>I need to do this because the command I want to use in the actual non-minimal-example program (<code>qacct -j</code>) has an output that's too big for <code>subprocess.PIPE</code>, raising a <code>MemoryError</code> (no further details).</p>

<p>The most relevant existing question I've found is <a href="http://stackoverflow.com/questions/2331339/piping-output-of-subprocess-popen-to-files">this</a>, but that didn't actually help.</p>

<p>The file is properly written to, but your <code>tfil.read()</code> command will start reading at the point where the last write happened, so it will return nothing. You need to call <code>tfil.seek(0)</code> first.</p>

python subprocess io-redirection temporary-files