Optimization of iterating over long list of string
<p>The below snippet of code executes in about 18s. <code>EdgeList</code> is estimated to be 330K in length. Is there a way to optimize it, considering that I am calling it multiple times.</p>

<p>I have tried optimizing the insertion into my <code>MeanspeedDict</code>. But I guess it's the best it can go already?     </p>

<pre><code>    EdgeList = traci.edge.getIDList() #Returns a list of Strings
    for edge in EdgeList:
        meanspeed = traci.edge.getLastStepMeanSpeed(edge) #returns a float value
        '''
        if edge in MeanspeedDict:
            MeanspeedDict[edge].append(meanspeed)
            MeanspeedDict[edge] = MeanspeedDict[edge][-300:] #Only keep the last 300 values
        else:
            MeanspeedDict[edge] = [meanspeed]
        '''
        try:
            MeanspeedDict[edge].append(meanspeed)
            MeanspeedDict[edge] = MeanspeedDict[edge][-300:] #Only keep the last 300 values
        except KeyError:
            MeanspeedDict[edge] = [meanspeed]
</code></pre>

<p>Profile Data as per request. Running it <strong>11</strong> times</p>

<pre><code>         252229348 function calls in 295.056 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000  295.056  295.056 &lt;string&gt;:1(&lt;module&gt;)
        1    0.000    0.000   40.158   40.158 __init__.py:101(getVersion)
        1    0.000    0.000    4.908    4.908 __init__.py:105(close)
        1    0.000    0.000    0.000    0.000 __init__.py:111(switch)
        1    0.000    0.000    0.507    0.507 __init__.py:45(connect)
        1    0.000    0.000    0.000    0.000 __init__.py:49(normalize_encoding)
        1    0.000    0.000   40.665   40.665 __init__.py:64(init)
        1    0.000    0.000    0.008    0.008 __init__.py:71(search_function)
       11    0.000    0.000    1.646    0.150 __init__.py:92(simulationStep)
  3607912    2.785    0.000  221.939    0.000 _edge.py:151(getLastStepMeanSpeed)
        1    0.000    0.000    0.000    0.000 codecs.py:92(__new__)
  3607923    5.689    0.000   16.796    0.000 connection.py:119(_beginMessage)
  3607923    3.451    0.000  210.529    0.000 connection.py:128(_sendReadOneStringCmd)
  3607923    8.231    0.000  190.282    0.000 connection.py:152(_checkResult)
       11    0.000    0.000    1.646    0.150 connection.py:254(simulationStep)
        1    0.000    0.000   40.158   40.158 connection.py:273(getVersion)
        1    0.000    0.000    4.908    4.908 connection.py:285(close)
        1    0.000    0.000    0.507    0.507 connection.py:48(__init__)
  3607923    4.167    0.000    8.645    0.000 connection.py:64(_packString)
  3607936   19.808    0.000  108.622    0.000 connection.py:72(_recvExact)
  3607936   19.507    0.000  200.505    0.000 connection.py:91(_sendExact)
       15    0.000    0.000    0.000    0.000 copy.py:123(_copy_inst)
       15    0.000    0.000    0.000    0.000 copy.py:66(copy)
       15    0.000    0.000    0.000    0.000 domain.py:108(_setConnection)
  3607923    4.273    0.000  236.643    0.000 domain.py:111(_getUniversal)
       11    0.004    0.000   17.493    1.590 domain.py:116(getIDList)
       15    0.000    0.000    0.000    0.000 domain.py:37(__init__)
      495    0.000    0.000    0.000    0.000 domain.py:47(reset)
       15    0.000    0.000    0.000    0.000 domain.py:99(_register)
        1    0.000    0.000    0.000    0.000 latin_1.py:13(Codec)
        1    0.000    0.000    0.000    0.000 latin_1.py:20(IncrementalEncoder)
        1    0.000    0.000    0.000    0.000 latin_1.py:24(IncrementalDecoder)
        1    0.000    0.000    0.000    0.000 latin_1.py:28(StreamWriter)
        1    0.000    0.000    0.000    0.000 latin_1.py:31(StreamReader)
        1    0.000    0.000    0.000    0.000 latin_1.py:34(StreamConverter)
        1    0.000    0.000    0.000    0.000 latin_1.py:41(getregentry)
        1    0.000    0.000    0.000    0.000 latin_1.py:8(&lt;module&gt;)
        1    0.000    0.000    0.000    0.000 six.py:180(find_module)
        1    0.000    0.000    0.000    0.000 six.py:184(find_module)
        1    0.001    0.001    0.001    0.001 socket.py:189(__init__)
        1    0.000    0.000    0.000    0.000 socket.py:196(close)
        2    0.000    0.000    0.506    0.253 socket.py:227(meth)
  3607936    1.590    0.000    1.590    0.000 storage.py:32(__init__)
 39687197   29.658    0.000   39.432    0.000 storage.py:36(read)
       12    0.000    0.000    0.000    0.000 storage.py:41(readInt)
  3607912    1.564    0.000    5.345    0.000 storage.py:44(readDouble)
  3607924    1.712    0.000    5.389    0.000 storage.py:47(readLength)
 10823772   17.543    0.000   50.373    0.000 storage.py:53(readString)
       11    1.922    0.175   16.496    1.500 storage.py:57(readStringList)
        1    0.000    0.000    0.000    0.000 subprocess.py:458(_cleanup)
        1    0.000    0.000    0.000    0.000 subprocess.py:578(list2cmdline)
        1    0.000    0.000    0.045    0.045 subprocess.py:651(__init__)
        1    0.000    0.000    0.000    0.000 subprocess.py:811(_get_handles)
        3    0.000    0.000    0.000    0.000 subprocess.py:881(_make_inheritable)
        1    0.000    0.000    0.045    0.045 subprocess.py:905(_execute_child)
        3    0.000    0.000    0.000    0.000 subprocess.py:946(_close_in_parent)
        1    6.670    6.670  295.056  295.056 traciTest_meanspeed.py:45(runTraCI)
        1    0.007    0.007    0.007    0.007 {__import__}
 39687197    3.728    0.000    3.728    0.000 {_struct.calcsize}
 10823795    3.374    0.000    3.374    0.000 {_struct.pack}
 43295133    7.225    0.000    7.225    0.000 {_struct.unpack}
        1    0.045    0.045    0.045    0.045 {_subprocess.CreateProcess}
        3    0.000    0.000    0.000    0.000 {_subprocess.DuplicateHandle}
        6    0.000    0.000    0.000    0.000 {_subprocess.GetCurrentProcess}
        1    0.000    0.000    0.000    0.000 {_subprocess.GetStdHandle}
        4    0.000    0.000    0.000    0.000 {built-in method Close}
        1    0.000    0.000    0.000    0.000 {built-in method __new__ of type object at 0x1E22B4F8}
        8    0.000    0.000    0.000    0.000 {getattr}
       61    0.000    0.000    0.000    0.000 {hasattr}
        6    0.000    0.000    0.000    0.000 {isinstance}
 32471566    2.246    0.000    2.246    0.000 {len}
        3    0.000    0.000    0.000    0.000 {method 'add' of 'set' objects}
 10496061    2.479    0.000    2.479    0.000 {method 'append' of 'list' objects}
      990    0.000    0.000    0.000    0.000 {method 'clear' of 'dict' objects}
        1    0.506    0.506    0.506    0.506 {method 'connect' of '_socket.socket' objects}
 10823772   12.557    0.000   12.565    0.000 {method 'decode' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
  3607923    3.371    0.000    3.371    0.000 {method 'encode' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {method 'fileno' of 'file' objects}
       17    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
        2    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
  7215956   84.503    0.000   84.503    0.000 {method 'recv' of '_socket.socket' objects}
        3    0.000    0.000    0.000    0.000 {method 'remove' of 'set' objects}
  3607936   46.414    0.000   46.414    0.000 {method 'send' of '_socket.socket' objects}
        1    0.000    0.000    0.000    0.000 {method 'setsockopt' of '_socket.socket' objects}
        1    0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {method 'startswith' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {method 'translate' of 'str' objects}
       15    0.000    0.000    0.000    0.000 {method 'update' of 'dict' objects}
       11    0.000    0.000    0.000    0.000 {method 'values' of 'dict' objects}
        2    0.000    0.000    0.000    0.000 {msvcrt.get_osfhandle}
       12    0.023    0.002    0.023    0.002 {range}
       27    0.000    0.000    0.000    0.000 {setattr}
        1    0.000    0.000    0.000    0.000 {sys.exit}
       22    0.000    0.000    0.000    0.000 {time.clock}
</code></pre>

<p>It seems <code>getLastStepMeanSpeed</code> is too slow:</p>

<pre><code>3607912    2.785    0.000  221.939    0.000 _edge.py:151(getLastStepMeanSpeed)
</code></pre>

<p>The script spend executing it 75% of the time (221/295)</p>

python python-2.7