Index into a Python iterator
<p>I have an iterator <code>iterator</code> and a list of indices <code>indices</code> (repeats possible) and I want to extract just those elements from my iterator. At the moment I'm doing</p>

<pre><code>indices = sorted(indices)
deltas = [indices[0]] + [indices[i+1] - indices[i] for i in range(len(indices) - 1)]
output = []
for delta in deltas:
    for i in range(delta):
        datum = next(iterator)
    output.append(datum)
</code></pre>

<p>Are those two layers of loop necessary? Am I missing a trick with <code>itertools</code>?</p>

<p>You definitely don't need the double loop as you can do this with a single loop and without creating deltas but the check code becomes more complicated:</p>

<pre><code>it = iter(sorted(indices))
index = next(it)
for i, datum in enumerate(iterator):
    if i != index:
        continue
    output.append(datum)
    try:
        index = next(it)
    except StopIteration:
        break
</code></pre>

<p>You can also do this in a list comprehension for very low number of indices as you incur overhead for the check (but avoid the <code>sort</code>):</p>

<pre><code>[datum for i, datum in enumerate(x) if i in indices]
</code></pre>

<p>You can reduce the cost of the check by converting <code>indices</code> to a <code>set</code>. I would be interested to see performance of <code>sort</code> over a <code>set</code> construction (set lookup is O(1)):</p>

<pre><code>indices = set(indices)
[datum for i, datum in enumerate(x) if i in indices]
</code></pre>

<p><strong>Update</strong>: First and third options are roughly equivalent in timing at just over 900 ms (slight edge to first) to choose 1000 random indices out of 10,000,000 items. The OP's code ran in about 1.2 s.</p>

python iterator itertools