List of lists: replacing and adding up items of sublists
<p>I have a list of lists, let's say something like this:</p>

<pre><code>tripInfo_csv = [['1','2',6,2], ['a','h',4,2], ['1','4',6,1], ['1','8',18,3], ['a','8',2,1]]
</code></pre>

<p>Think of sublists as trips: [start point, end point, number of adults, number of children]</p>

<p>My aim is to get a list where trips with coincident start and end points get their third and fourth values added up. The start and end values should always be numbers from 1 to lets say 8. If they are letters instead, those should be replaced with the corresponding number (a=1, b=2, and so on).</p>

<p>This is my code. It works but I'm sure it can be improved. The main issue for me is performance. I have quite a number of lists like this with many more sublists.</p>

<pre><code>dicPoints = {'a':'1','b':'2','c':'3', 'd':'4', 'e':'5', 'f':'6', 'g':'7', 'h':'8'}
def getTrips (trips):
    okTrips = []
    for trip in trips:
        if not trip[0].isdigit():
            trip[0] = dicPoints[trip[0]]
        if not trip[1].isdigit():
            trip[1] = dicPoints[trip[1]]

        if len(okTrips) == 0:
            okTrips.append(trip)
        else:
            for i, stop in enumerate(okTrips):
                if stop[0] == trip[0] and stop[1] == trip[1]:
                    stop[2] += trip[2]
                    stop[3] += trip[3]
                    break
                else:
                    if i == len(okTrips)-1:
                        okTrips.append(trip)
</code></pre>

<p><strong>As <em>eguaio</em> mentioned the code above has a bug. It should be like this:</strong></p>

<pre><code>def getTrips (trips):
    okTrips = []
    print datetime.datetime.now()
    for trip in trips:
        if not trip[0].isdigit():
            trip[0] = dicPoints[trip[0]]
        if not trip[1].isdigit():
            trip[1] = dicPoints[trip[1]]

        if len(okTrips) == 0:
            okTrips.append(trip)
        else:
            flag = 0
            for i, stop in enumerate(okTrips):
                if stop[0] == trip[0] and stop[1] == trip[1]:
                    stop[2] += trip[2]
                    stop[3] += trip[3]
                    flag = 1
                    break

            if flag == 0:
                okTrips.append(trip)
</code></pre>

<hr>

<p><strong>I got an improved version thanks to eguaio's answer that I want to share. This is my script based on his answer.
My data and requirements are more complex now than what I was first told so I made a few changes.</strong></p>

<p>CSV files look like this:</p>

<pre><code>LineT;Line;Route;Day;Start_point;End_point;Adults;Children;First_visit
SM55;5055;3;Weekend;15;87;21;4;0 
SM02;5002;8;Weekend;AF3;89;5;0;1 
...
</code></pre>

<p>Script:</p>

<pre><code>import os, csv, psycopg2

folder = "F:/route_project/routes"

# Day type
dicDay = {'Weekday':1,'Weekend':2,'Holiday':3}

# Dictionary with the start and end points of each route
#  built from a Postgresql table (with coumns: line_route, start, end)
conn = psycopg2.connect (database="test", user="test", password="test", host="###.###.#.##")
cur = conn.cursor()
cur.execute('select id_linroute, start_p, end_p from route_ends')
recs = cur.fetchall()
dicPoints = {rec[0]: rec[1:] for rec in recs}

# When point labels are text, replace them with a number label in dicPoints
# Text is not important: they are special text labels for start and end
#  of routes (for athletes), so we replace them with labels for start or
#  the end of each route
def convert_point(line, route, point, i):
    if point.isdigit():
        return point
    else:
        return dicPoints["%s_%s" % (line,route)][i]

# Points with text labels mean athletes made the whole or part of this route,
#  we keep them as adults but also keep this number as an extra value
#  for further purposes
def num_athletes(start_p, end_p, adults):
    if not start_p.isdigit() or not end_p.isdigit():
        return adults
    else:
        return 0

# Data is taken for CSV files in subfolders
for root, dirs, files in os.walk(folder):
    for file in files:
        if file.endswith(".csv"):
            file_path = (os.path.join(root, file))
            with open(file_path, 'rb') as csvfile:
                rows = csv.reader(csvfile, delimiter=';', quotechar='"')
                # Skips the CSV header row
                rows.next()
                # linT is not used, yet it's found in every CSV file
                # There's an unused last column in every file, I take advantage out of it
                #  to store the number of athletes in the generator
                gen =((lin, route, dicDay[tday], convert_point(lin,route,s_point,0), convert_point(lin,route,e_point,1), adults, children, num_athletes(s_point,e_point,adults)) for linT, lin, route, tday, s_point, e_point, adults, children, athletes in rows)
                dicCSV = {}
                for lin, route, tday, s_point, e_point, adults, children, athletes in gen:
                    visitors = dicCSV.get(("%s_%s_%s" % (lin,route,s_point), "%s_%s_%s" % (lin,route,e_point), tday), (0, 0, 0))
                    dicCSV[("%s_%s_%s" % (lin,route,s_point), "%s_%s_%s" % (lin,route,e_point), tday)] = (visitors[0] + int(adults), visitors[1] + int(children), visitors[2] + int(athletes))

for k,v in dicCSV.iteritems():
    print k, v
</code></pre>

<p>To handle this more efficiently it's best to sort the input list by the start and end points, so that rows which have matching start and end points are grouped together. Then we can easily use the <code>groupby</code> function to process those groups efficiently.</p>

<pre><code>from operator import itemgetter
from itertools import groupby

tripInfo_csv = [
    ['1', '2', 6, 2], 
    ['a', 'h', 4, 2], 
    ['1', '4', 6, 1], 
    ['1', '8', 18, 3], 
    ['a', '8', 2, 1],
]

# Used to convert alphabetic point labels to numeric form
dicPoints = {v:str(i) for i, v in enumerate('abcdefgh', 1)}

def fix_points(seq):
    return [dicPoints.get(p, p) for p in seq]

# Ensure that all point labels are numeric
for row in tripInfo_csv:
    row[:2] = fix_points(row[:2])

# Sort on point labels
keyfunc = itemgetter(0, 1)
tripInfo_csv.sort(key=keyfunc)

# Group on point labels and sum corresponding adult &amp; child numbers
newlist = []
for k, g in groupby(tripInfo_csv, key=keyfunc):
    g = list(g)
    row = list(k) + [sum(row[2] for row in g), sum(row[3] for row in g)]
    newlist.append(row)

# Print the condensed list
for row in newlist:
    print(row)
</code></pre>

<p><strong>output</strong></p>

<pre><code>['1', '2', 6, 2]
['1', '4', 6, 1]
['1', '8', 24, 6]
</code></pre>

python nested-lists