A print function makes a multiprocessing program fail
<p>In the following code, I'm trying to create a sandboxed master-worker system, in which changes to global variables in a worker don't reflect to other workers.</p>

<p>To achieve this, a new process is created each time a task is created, and to make the execution parallel, the creation of processes itself is managed by <code>ThreadPoolExecutor</code>.</p>

<pre><code>import time
from concurrent.futures import ThreadPoolExecutor
from multiprocessing import Pipe, Process


def task(conn, arg):
  conn.send(arg * 2)


def isolate_fn(fn, arg):

  def wrapped():
    parent_conn, child_conn = Pipe()
    p = Process(target=fn, args=(child_conn, arg), daemon=True)
    try:
      p.start()
      r = parent_conn.recv()
    finally:
      p.join()
    return r

  return wrapped


def main():
  with ThreadPoolExecutor(max_workers=4) as executor:
    pair = []

    for i in range(0, 10):
      pair.append((i, executor.submit(isolate_fn(task, i))))

      # This function makes the program broken.
      # 
      print('foo')

    time.sleep(2)

    for arg, future in pair:
      if future.done():
        print('arg: {}, res: {}'.format(arg, future.result()))
      else:
        print('not finished: {}'.format(arg))

  print('finished')

main()
</code></pre>

<p>This program works fine, until I put the <code>print('foo')</code> function inside the loop.  If the function exists, some tasks remain unfinished, and what is worse, this program itself doesn't finish.</p>

<p>Results are not always the same, but the following is the typical output:</p>

<pre><code>foo
foo
foo
foo
foo
foo
foo
foo
foo
foo
arg: 0, res: 0
arg: 1, res: 2
arg: 2, res: 4
not finished: 3
not finished: 4
not finished: 5
not finished: 6
not finished: 7
not finished: 8
not finished: 9
</code></pre>

<p>Why is this program so fragile?</p>

<p>I use Python 3.4.5.</p>

<p>You are not creating ThreadPoolExecutor every time , rather using the pre initialized pool for every iteration. I really not able to track which print statement is hindering you?</p>

python python-3.x python-multithreading python-multiprocessing