Python Multiprocessing, Pool map - cancel all running processes if one, returns the desired result
<p>given the following Python code:</p>

<pre><code>import multiprocessing

def unique(somelist):
    return len(set(somelist)) == len(somelist)


if __name__ == '__main__':
    somelist = [[1,2,3,4,5,6,7,8,9,10,11,12,13,2], [1,2,3,4,5], [1,2,3,4,5,6,7,8,9,1], [0,1,5,1]]

    pool = multiprocessing.Pool()
    reslist = pool.map(unique, somelist)
    pool.close()
    pool.join()
    print "Done!"

    print reslist
</code></pre>

<p>Now imagine, that the lists with integers in this toy example are extremely long, and what I'd like to achieve here is the following: if one of the lists in somelist returns True, kill all running processes. </p>

<p>This leads to two questions (and probably more which I haven't come up with):</p>

<ul>
<li><p>How can I 'read'/'listen' from a finished process the result, while other processes are running? If e.g. a process is dealing with [1,2,3,4,5] from somelist, and is finished before all other processes, how can I read out the result from that process in this very moment?</p></li>
<li><p>Given the case that it is possible to 'read' out the result of a finished process while other are running: how can I use this result as a condition to terminate all other running processes?</p></li>
</ul>

<p>e.g. if one process has finished and returned True, how I can use this as a condition to terminate all other (still) running processes?</p>

<p>Thank you in advance for any hints
Dan</p>

<p>Use <code>pool.imap_unordered</code> to view the results in any order they come up.</p>

<pre><code>reslist = pool.imap_unordered(unique, somelist)
pool.close()
for res in reslist:
    if res:  # or set other condition here
        pool.terminate()
        break
pool.join()
</code></pre>

<p>You can iterate over an <code>imap</code> reslist in your main process while the pool processes are still generating results.</p>

python dictionary multiprocessing pool