List of lists: replacing and adding up items of sublists
<p>I have a list of lists, let's say something like this:</p>

<pre><code>tripInfo_csv = [['1','2',6,2], ['a','h',4,2], ['1','4',6,1], ['1','8',18,3], ['a','8',2,1]]
</code></pre>

<p>Think of sublists as trips: [start point, end point, number of adults, number of children]</p>

<p>My aim is to get a list where trips with coincident start and end points get their third and fourth values added up. The start and end values should always be numbers from 1 to lets say 8. If they are letters instead, those should be replaced with the corresponding number (a=1, b=2, and so on).</p>

<p>This is my code. It works but I'm sure it can be improved. The main issue for me is performance. I have quite a number of lists like this with many more sublists.</p>

<pre><code>dicPoints = {'a':'1','b':'2','c':'3', 'd':'4', 'e':'5', 'f':'6', 'g':'7', 'h':'8'}
def getTrips (trips):
    okTrips = []
    for trip in trips:
        if not trip[0].isdigit():
            trip[0] = dicPoints[trip[0]]
        if not trip[1].isdigit():
            trip[1] = dicPoints[trip[1]]

        if len(okTrips) == 0:
            okTrips.append(trip)
        else:
            for i, stop in enumerate(okTrips):
                if stop[0] == trip[0] and stop[1] == trip[1]:
                    stop[2] += trip[2]
                    stop[3] += trip[3]
                    break
                else:
                    if i == len(okTrips)-1:
                        okTrips.append(trip)
</code></pre>

<p><strong>As <em>eguaio</em> mentioned the code above has a bug. It should be like this:</strong></p>

<pre><code>def getTrips (trips):
    okTrips = []
    print datetime.datetime.now()
    for trip in trips:
        if not trip[0].isdigit():
            trip[0] = dicPoints[trip[0]]
        if not trip[1].isdigit():
            trip[1] = dicPoints[trip[1]]

        if len(okTrips) == 0:
            okTrips.append(trip)
        else:
            flag = 0
            for i, stop in enumerate(okTrips):
                if stop[0] == trip[0] and stop[1] == trip[1]:
                    stop[2] += trip[2]
                    stop[3] += trip[3]
                    flag = 1
                    break

            if flag == 0:
                okTrips.append(trip)
</code></pre>

<hr>

<p><strong>I got an improved version thanks to eguaio's answer that I want to share. This is my script based on his answer.
My data and requirements are more complex now than what I was first told so I made a few changes.</strong></p>

<p>CSV files look like this:</p>

<pre><code>LineT;Line;Route;Day;Start_point;End_point;Adults;Children;First_visit
SM55;5055;3;Weekend;15;87;21;4;0 
SM02;5002;8;Weekend;AF3;89;5;0;1 
...
</code></pre>

<p>Script:</p>

<pre><code>import os, csv, psycopg2

folder = "F:/route_project/routes"

# Day type
dicDay = {'Weekday':1,'Weekend':2,'Holiday':3}

# Dictionary with the start and end points of each route
#  built from a Postgresql table (with coumns: line_route, start, end)
conn = psycopg2.connect (database="test", user="test", password="test", host="###.###.#.##")
cur = conn.cursor()
cur.execute('select id_linroute, start_p, end_p from route_ends')
recs = cur.fetchall()
dicPoints = {rec[0]: rec[1:] for rec in recs}

# When point labels are text, replace them with a number label in dicPoints
# Text is not important: they are special text labels for start and end
#  of routes (for athletes), so we replace them with labels for start or
#  the end of each route
def convert_point(line, route, point, i):
    if point.isdigit():
        return point
    else:
        return dicPoints["%s_%s" % (line,route)][i]

# Points with text labels mean athletes made the whole or part of this route,
#  we keep them as adults but also keep this number as an extra value
#  for further purposes
def num_athletes(start_p, end_p, adults):
    if not start_p.isdigit() or not end_p.isdigit():
        return adults
    else:
        return 0

# Data is taken for CSV files in subfolders
for root, dirs, files in os.walk(folder):
    for file in files:
        if file.endswith(".csv"):
            file_path = (os.path.join(root, file))
            with open(file_path, 'rb') as csvfile:
                rows = csv.reader(csvfile, delimiter=';', quotechar='"')
                # Skips the CSV header row
                rows.next()
                # linT is not used, yet it's found in every CSV file
                # There's an unused last column in every file, I take advantage out of it
                #  to store the number of athletes in the generator
                gen =((lin, route, dicDay[tday], convert_point(lin,route,s_point,0), convert_point(lin,route,e_point,1), adults, children, num_athletes(s_point,e_point,adults)) for linT, lin, route, tday, s_point, e_point, adults, children, athletes in rows)
                dicCSV = {}
                for lin, route, tday, s_point, e_point, adults, children, athletes in gen:
                    visitors = dicCSV.get(("%s_%s_%s" % (lin,route,s_point), "%s_%s_%s" % (lin,route,e_point), tday), (0, 0, 0))
                    dicCSV[("%s_%s_%s" % (lin,route,s_point), "%s_%s_%s" % (lin,route,e_point), tday)] = (visitors[0] + int(adults), visitors[1] + int(children), visitors[2] + int(athletes))

for k,v in dicCSV.iteritems():
    print k, v
</code></pre>

<p>Let say start and end points are between 0 and n values. </p>

<p>Then, the result 'OkTrip' has maximum n^2 elements. Then, your second loop in function has a complexity O(n^2). It is possible to reduce the complexity to O(n) if you have not problem with space complexity.</p>

<p>Firslty, create dict which contains n lists such that k'(th) sublist contains trips starting with 'k'. </p>

<p>When you search whether there are different trips with same start and end points, you need search only corresponding sublist instead of searching all elements. </p>

<p>The idea comes from sparse matrix storage techniques.
I could not check validation of following code. </p>

<p>The code is following,</p>

<pre><code>dicPoints = {'a':'1','b':'2','c':'3', 'd':'4', 'e':'5', 'f':'6', 'g':'7', 'h':'8'}
Temp = {'1':[],'2':[],'3':[],'4':[],'5':[],'6':[],'7':[],'8':[]};
def getTrips (trips):
   okTrips = []
   for trip in trips:
        if not trip[0].isdigit():
            trip[0] = dicPoints[trip[0]]
        if not trip[1].isdigit():
            trip[1] = dicPoints[trip[1]]

        if len(Temp[trip[0]]) == 0:
            Temp[trip[0]].append(trip)
        else:
            for i, stop in enumerate(Temp[trip[0]]):
                if stop[1] == trip[1]:
                   stop[2] += trip[2]
                   stop[3] += trip[3]
                   break
                else:
                   if i == len(Temp[trip[0]])-1:
                       Temp[trip[0]].append(trip)
        print Temp

    for key in Temp:
        okTrips = okTrips + Temp[key];
</code></pre>

python nested-lists