Build 2 lists in one go while reading from file, pythonically
<p>I'm reading a big file with hundreds of thousands of number pairs representing the edges of a graph. I want to build 2 lists as I go: one with the forward edges and one with the reversed. </p>

<p>Currently I'm doing an explicit <code>for</code> loop, because I need to do some pre-processing on the lines I read. However, I'm wondering if there is a more pythonic approach to building those lists, like list comprehensions, etc. </p>

<p>But, as I have 2 lists, I don't see a way to populate them using comprehensions without reading the file twice.</p>

<p>My code right now is:</p>

<pre><code>with open('SCC.txt') as data:
    for line in data:
        line = line.rstrip()
        if line:
            edge_list.append((int(line.rstrip().split()[0]), int(line.rstrip().split()[1])))
            reversed_edge_list.append((int(line.rstrip().split()[1]), int(line.rstrip().split()[0])))
</code></pre>

<p>I would keep your logic as it is the <em>Pythonic</em> approach just not <em>split/rstrip</em> the same line multiple times:</p>

<pre><code>with open('SCC.txt') as data:
    for line in data:
        spl = line.split()
        if spl:
            i, j = map(int, spl)
            edge_list.append((i, j))
            reversed_edge_list.append((j, i))
</code></pre>

<p>Calling <em>rstrip</em> when you have already called it is  redundant in itself even more so when you are splitting as that would already remove the whitespace so splitting just once means you save doing a lot of unnecessary work.</p>

<p>You can also use <em>csv.reader</em> to read the data and filter empty rows once you have a single whitespace delimiting:</p>

<pre><code>from csv import reader

with open('SCC.txt') as data:
    edge_list, reversed_edge_list = [], []
    for i, j in filter(None, reader(data, delimiter=" ")):
        i, j = int(i), int(j)
        edge_list.append((i, j))
        reversed_edge_list.append((j, i))
</code></pre>

<p>Or if there are multiple whitespaces delimiting you can use <code>map(str.split, data)</code>:</p>

<pre><code>    for i, j in filter(None, map(str.split, data)):
        i, j = int(i), int(j)
</code></pre>

<p>Whatever you choose will be faster than going over the data twice or splitting the sames lines multiple times.</p>

python list python-3.x