Python multiprocess lists of images
<p>I want to use multi process to stack many images. Each stack consists of 5 images, which means I have a list of images with a sublist of the images which should be combined:</p>

<blockquote>
  <p>img_lst = [[01_A, 01_B, 01_C, 01_D, 01_E], [02_A, 02_B, 02_C, 02_D, 02_E], [03_A, 03_B, 03_C, 03_D, 03_E]]</p>
</blockquote>

<p>At them moment I call my function do_stacking(sub_lst) with a loop:</p>

<pre><code>for sub_lst in img_lst:
    # example: do_stacking([01_A, 01_B, 01_C, 01_D, 01_E])
    do_stacking(sub_lst)
</code></pre>

<p>I want to speed up with multiprocessing but I am not sure how to call pool.map function:</p>

<pre><code>if __name__ == '__main__':
    from multiprocessing import Pool

    # I store my lists in a file
    f_in = open(stacking_path + "stacks.txt", 'r') 
    f_stack = f_in.readlines()

    for data in f_stack:
        data = data.strip()
        data = data.split('\t')
        # data is now my sub_lst

    # Not sure what to do here, set the sublist, f_stack?
    pool = Pool()
    pool.map(do_stacking, ???)
    pool.close()
    pool.join()
</code></pre>

<p><strong>Edit</strong>:</p>

<p>I have a list of list:</p>

<p>[</p>

<p>[01_A, 01_B, 01_C, 01_D, 01_E],</p>

<p>[02_A, 02_B, 02_C, 02_D, 02_E], </p>

<p>[03_A, 03_B, 03_C, 03_D, 03_E]</p>

<p>]</p>

<p>Each sublist should be passed to a function called do_stacking(sublist). I only want to proceed with the sublist and not with the entire list.</p>

<p>My question is how to handle the loop of the list (for x in img_lst)? Should I create a loop for each Pool?</p>

<p><code>Pool.map</code> works like the builtin <code>map</code> function.It fetch one element from the second argument each time and pass it to the function that represent by the first argument.</p>

<pre><code>if __name__ == '__main__':
    from multiprocessing import Pool

    # I store my lists in a file
    f_in = open(stacking_path + "stacks.txt", 'r') 
    f_stack = f_in.readlines()
    img_list = []
    for data in f_stack:
        data = data.strip()
        data = data.split('\t')
        # data is now my sub_lst
        img_list.append(data)
        print img_list # check if the img_list is right?

    # Not sure what to do here, set the sublist, f_stack?
    pool = Pool()
    pool.map(do_stacking, img_list)
    pool.close()
    pool.join()
</code></pre>

python multiprocessing