multiprocessing function call
<p>I am trying to do some multiprocessing in Python. I have a function doing some work and returning a list. I want to repeat that for several cases. Finally, I want to get the returned list of each parallel call and unify them (have only one list with all duplicates removed).</p>

<pre><code>def get_version_list(env):
    list = []
    #do some intensive work
    return list

from multiprocessing import Pool

pool = Pool()

result1 = pool.apply_async(get_version_list, ['prod'])
result2 = pool.apply_async(get_version_list, ['uat'])
#etc, I have six environment to check.

alist = result1.get()
blist = result2.get()
</code></pre>

<p>This does not work (not sure about the function call syntax, but I tried other things too, without success), it gives me an error (and repeats it a lot since my intensive work is doing around 300 request.post calls).</p>

<p>RuntimeError:
    Attempt to start a new process before the current process
    has finished its bootstrapping phase.</p>

<pre><code>This probably means that you are on Windows and you have
forgotten to use the proper idiom in the main module:

    if __name__ == '__main__':
        freeze_support()
        ...

The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce a Windows executable.
</code></pre>

<p>You have to put the multiprocessing portion inside of your main function, like:</p>

<pre><code>def get_version_list(env):
    list = []
    print "ENV: " + env
    return list


if __name__ == '__main__':
    from multiprocessing import Pool

    pool = Pool()

    result1 = pool.apply_async(get_version_list, ['prod'])
    result2 = pool.apply_async(get_version_list, ['uat'])
    #etc, I have six environment to check.

    alist = result1.get()
    blist = result2.get()
</code></pre>

python multiprocessing