How to use yaml.load_all with fileinput.input?
<p>Without resorting to <code>''.join</code>, is there a Pythonic way to use PyYAML's <code>yaml.load_all</code> with <code>fileinput.input()</code> for easy streaming of multiple documents from multiple sources?</p>

<p>I'm looking for something like the following (non-working example):</p>

<pre><code># example.py
import fileinput

import yaml

for doc in yaml.load_all(fileinput.input()):
    print(doc)
</code></pre>

<p>Expected output:</p>

<pre><code>$ cat &gt;pre.yaml &lt;&lt;&lt;'--- prefix-doc'
$ cat &gt;post.yaml &lt;&lt;&lt;'--- postfix-doc'
$ python example.py pre.yaml - post.yaml &lt;&lt;&lt;'--- hello'
prefix-doc
hello
postfix-doc
</code></pre>

<p>Of course, <code>yaml.load_all</code> expects either a string, bytes, or a file-like object and <code>fileinput.input()</code> is none of those things, so the above example does not work.</p>

<p>Actual output:</p>

<pre><code>$ python example.py pre.yaml - post.yaml &lt;&lt;&lt;'--- hello'
...
AttributeError: FileInput instance has no attribute 'read'
</code></pre>

<p>You can make the example work with <code>''.join</code>, but that's cheating. I'm looking for a way that does not read the entire stream into memory at once.</p>

<p>We might rephrase the question as <em>Is there some way to emulate a string, bytes, or file-like object that proxies to an underlying iterator of strings?</em> However, I doubt that <code>yaml.load_all</code> actually needs the entire file-like interface, so that phrasing would ask for more than is strictly necessary.</p>

<p>Ideally I'm looking for the minimal adapter that would support something like this:</p>

<pre><code>for doc in yaml.load_all(minimal_adapter(fileinput.input())):
    print(doc)
</code></pre>

<p>The problem with <code>fileinput.input</code> is that the resulting object doesn't have a <code>read</code> method, which is what <code>yaml.load_all</code> is looking for.  If you're willing to give up <code>fileinput</code>, you can just write your own class that will do what you want:</p>

<pre><code>import sys                                                                      
import yaml                                                                     

class BunchOFiles (object):                                                     
    def __init__(self, *files):                                                 
        self.files = files                                                      
        self.fditer = self._fditer()                                            
        self.fd = self.fditer.next()                                            

    def _fditer(self):                                                          
        for fn in self.files:                                                   
            with sys.stdin if fn == '-' else open(fn, 'r') as fd:               
                yield fd                                                        

    def read(self, size=-1):                                                    
        while True:                                                             
            data = self.fd.read(size)                                           

            if data:                                                            
                break                                                           
            else:                                                               
                try:                                                            
                    self.fd = self.fditer.next()                                
                except StopIteration:                                           
                    self.fd = None                                              
                    break                                                       

        return data                                                             

bunch = BunchOFiles(*sys.argv[1:])                                              
for doc in yaml.load_all(bunch):                                                
    print doc                                                                   
</code></pre>

<p>The <code>BunchOFiles</code> class gets you an object with a <code>read</code> method that will happily iterate over a list of files until everything is exhausted.  Given the above code and your sample input, we get exactly the output you're looking for.</p>

python pyyaml