How to make huge loops in python faster on mac?
<p>I am a computer science student and some of the things I do require me to run huge loops on Macbook with dual core i5. Some the loops take 5-6 hours to complete but they only use 25% of my CPU. Is there a way to make this process faster? I cant change my loops but is there a way to make them run faster?</p>

<p>Thank you</p>

<p>Mac OS 10.11
Python 2.7 (I have to use 2.7) with IDLE or Spyder on Anaconda</p>

<p>Here is a sample code that takes 15 minutes:</p>

<pre><code>def test_false_pos():
    sumA = [0] * 1000
    for test in range(1000):
        counter = 0
        bf = BloomFilter(4095,10)
        for i in range(600):
            bf.rand_inserts()
        for x in range(10000):
            randS = str(rnd.randint(0,10**8))
            if bf.lookup(randS):
                counter += 1
        sumA[test] = counter/10000.0
    avg = np.mean(sumA)
    return avg
</code></pre>

<p>Consider looking into Numba and Jit (just in time compiler).  It works for functions that are Numpy based.  It can handle some python routines, but is mainly for speeding up numerical calculations, especially ones with loops (like doing cholesky rank-1 up/downdates).  I don't think it would work with a BloomFilter, but it is generally super helpful to know about. </p>

<p>In cases where you must use other packages in your flow with numpy, separate out the heavy-lifting numpy routines into their own functions, and throw a @jit decorator on top of that function.  Then put them into your flows with normal python stuff.  </p>

python performance python-2.7 loops cpu-usage