Why can generator expressions be iterated over only once?
<p>This program:</p>

<pre><code>def pp(seq):
    print(''.join(str(x) for x in seq))
    print(''.join(str(x) for x in seq))
    print('---')

pp([0,1,2,3])
pp(range(4))  # range in Python3, xrange in Python2
pp(x for x in [0,1,2,3])
</code></pre>

<p>prints this:</p>

<pre><code>0123
0123
---
0123
0123
---
0123

---
</code></pre>

<p>It is a result of the difference between <code>container.__iter__</code> and <code>iterator.__iter__</code>. Both are documented here: <a href="https://docs.python.org/3/library/stdtypes.html#typeiter" rel="nofollow">https://docs.python.org/3/library/stdtypes.html#typeiter</a>. If the <code>__iter__</code> returns a new iterator each time, we see two repeating lines. And if it returns the same iterator each time, we see just one line, beacuse the iterator is exhausted after that.</p>

<p>My question is <strong>why it was decided to implement generator expressions not similar to other objects that seem to be equivalent or at least very similar?</strong></p>

<p>Here is another example that generator expressions are different than other similar types which may lead to unexpected outcome.</p>

<pre><code>def pp(seq):
    # accept only non-empty sequences
    if seq:
        print("data ok")
    else:
        print("required data missing")

pp([])
pp(range(0))
pp(x for x in []) 
</code></pre>

<p>output:</p>

<pre><code>required data missing
required data missing
data ok
</code></pre>

<p>A generator <em>runs arbitrary code</em>, and returns a lazy sequence with the items yielded by that code.</p>

<ul>
<li>That code could be providing an infinite sequence.</li>
<li>That code could be reading contents off a network connection.</li>
<li>That code could be modifying external variables every time it iterates.</li>
</ul>

<p>Thus, you can't cache the results safely in the general case: If it's an infinite sequence, you'd run out of memory.</p>

<p>You also can't simply rerun the code when it's read from again: If it's reading off a network connection, then the connection may <em>no longer be there</em> again in the future (or may be in a different state). Similarly, if the code is modifying variables outside the genexp's scope, then that state would be changed in hard-to-predict ways based on the readers' behavior -- an undesirable propery for a language that values predictability and readability.</p>

<hr>

<p>Some other languages (notably Clojure) <em>do</em> implement generators (there, generalized as "lazy sequences") that cache results if and only if a reference to the beginning of the sequence is retained. This gives the programmer control, but at a cost of complexity: You need to know exactly what is and is not referenced for the garbage collector.</p>

<p>The decision not to do this for Python is entirely reasonable and in keeping with the language's design goals.</p>

python