Build 2 lists in one go while reading from file, pythonically
<p>I'm reading a big file with hundreds of thousands of number pairs representing the edges of a graph. I want to build 2 lists as I go: one with the forward edges and one with the reversed. </p>

<p>Currently I'm doing an explicit <code>for</code> loop, because I need to do some pre-processing on the lines I read. However, I'm wondering if there is a more pythonic approach to building those lists, like list comprehensions, etc. </p>

<p>But, as I have 2 lists, I don't see a way to populate them using comprehensions without reading the file twice.</p>

<p>My code right now is:</p>

<pre><code>with open('SCC.txt') as data:
    for line in data:
        line = line.rstrip()
        if line:
            edge_list.append((int(line.rstrip().split()[0]), int(line.rstrip().split()[1])))
            reversed_edge_list.append((int(line.rstrip().split()[1]), int(line.rstrip().split()[0])))
</code></pre>

<p>Here comes a solution</p>

<p>A test file:</p>

<pre><code>In[19]: f = ["{} {}".format(i,j) for i,j in zip(xrange(10), xrange(10, 20))]
In[20]: f
Out[20]: 
['0 10',
 '1 11',
 '2 12',
 '3 13',
 '4 14',
 '5 15',
 '6 16',
 '7 17',
 '8 18',
 '9 19']
</code></pre>

<p>One liner using comprehension, zip and map:</p>

<pre><code>In[27]: l, l2 = map(list,zip(*[(tuple(map(int, x.split())), tuple(map(int, x.split()))[::-1]) for x in f]))
In[28]: l
Out[28]: 
[(0, 10),
 (1, 11),
 (2, 12),
 (3, 13),
 (4, 14),
 (5, 15),
 (6, 16),
 (7, 17),
 (8, 18),
 (9, 19)]
In[29]: l2
Out[29]: 
[(10, 0),
 (11, 1),
 (12, 2),
 (13, 3),
 (14, 4),
 (15, 5),
 (16, 6),
 (17, 7),
 (18, 8),
 (19, 9)]
</code></pre>

<p>Explaining, with <code>[(tuple(map(int, x.split())), tuple(map(int, x.split()))[::-1]) for x in f]</code> we build a list containing a pair tuple with the pair tuples and its reversed forms:</p>

<pre><code>In[24]: [(tuple(map(int, x.split())), tuple(map(int, x.split()))[::-1]) for x in f]
Out[24]: 
[((0, 10), (10, 0)),
 ((1, 11), (11, 1)),
 ((2, 12), (12, 2)),
 ((3, 13), (13, 3)),
 ((4, 14), (14, 4)),
 ((5, 15), (15, 5)),
 ((6, 16), (16, 6)),
 ((7, 17), (17, 7)),
 ((8, 18), (18, 8)),
 ((9, 19), (19, 9))]
</code></pre>

<p>Applaying <code>zip</code> to the unpack form we split the tuples inside the main tuple, so we have 2 tuples containing the tuples pairs in the first and the reversed in the others:</p>

<pre><code>In[25]: zip(*[(tuple(map(int, x.split())), tuple(map(int, x.split()))[::-1]) for x in f])
Out[25]: 
[((0, 10),
  (1, 11),
  (2, 12),
  (3, 13),
  (4, 14),
  (5, 15),
  (6, 16),
  (7, 17),
  (8, 18),
  (9, 19)),
 ((10, 0),
  (11, 1),
  (12, 2),
  (13, 3),
  (14, 4),
  (15, 5),
  (16, 6),
  (17, 7),
  (18, 8),
  (19, 9))]
</code></pre>

<p>Almost there, we just use <code>map</code> to transform that tuples into lists.</p>

<p><strong>EDIT:</strong>
as @PadraicCunningham asked, for filtering empty lines, just add a <code>if x</code> in the comprehension <code>[ ... for x in f if x]</code></p>

python list python-3.x