Python Multiprocessing data output wrong
<p>I am trying Multiprocessing in Python. I have written some code which does vector add, but couldn't get the output out of the function. Which mean, the output Z prints out 0 rather than 2.</p>

<pre><code>from multiprocessing import Process
import numpy as np

numThreads = 16
num = 16

numIter = num/numThreads

X = np.ones((num, 1))
Y = np.ones((num, 1))
Z = np.zeros((num, 1))

def add(X,Y,Z,j):
    Z[j] = X[j] + Y[j]

if __name__ == '__main__':
  jobs = []
  for i in range(numThreads):
    p = Process(target=add, args=(X, Y, Z, i,))
    jobs.append(p)

  for i in range(numThreads):
    jobs[i].start()

  for i in range(numThreads):
    jobs[i].join()

  print Z[0]
</code></pre>

<p>Edit: Took advice of clocker, and changed my code to this:</p>

<pre><code>import multiprocessing
import numpy as np

numThreads = 16
numRows = 32000
numCols = 2
numOut = 3

stride = numRows / numThreads

X = np.ones((numRows, numCols))
W = np.ones((numCols, numOut))
B = np.ones((numRows, numOut))
Y = np.ones((numRows, numOut))

def conv(idx):
  Y[idx*stride:idx*stride+stride] = X[idx*stride:idx*stride+stride].dot(W) + B[idx*stride:idx*stride+stride]

if __name__=='__main__':
  pool = multiprocessing.Pool(numThreads)
  pool.map(conv, range(numThreads))
  print Y
</code></pre>

<p>And the output is Y instead of a Saxp.</p>

<p>The reason your last line <code>print Z[0]</code> returns [0] instead of [2] is that each of the processes makes an independent copy of Z (or may be <code>Z[j]</code> - not completely sure about that) before modifying it. Either way, a separate process run will guarantee that your original version will be unchanged.</p>

<p>If you were to use the <code>threading module</code> instead the last line would indeed return [2] as expected, but that is not multi-processing. </p>

<p>So, you probably want to use <code>multiprocessing.Pool</code> instead. Going along your experiment purely for illustration, one could do the following:</p>

<pre><code>In [40]: pool = multiprocessing.Pool()
In [41]: def add_func(j):
   ....:     return X[j] + Y[j]
In [42]: pool = multiprocessing.Pool(numThreads)
In [43]: pool.map(add_func, range(numThreads))
Out[43]: 
[array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.]),
 array([ 2.])]
</code></pre>

<p>Have fun!</p>

<p>For your second part to your question, the problem is that the conv() function does not return any value. While the process pool gets a copy of X, B and W for pulling values from, the Y inside conv() is local to each process that is launched. To get the new computed value of Y, you would use something like this:</p>

<pre><code>def conv(idx):
    Ylocal_section = X[idx*stride:idx*stride+stride].dot(W) +  B[idx*stride:idx*stride+stride]
    return Ylocal_section 

results = pool.map(conv, range(numThreads)) # then apply each result to Y
for idx in range(numThreads):
    Y[idx*stride:idx*stride+stride] = results[idx] 
</code></pre>

<p>Parallelism can get complicated really fast, and at this point I would evaluate existing libraries that can perform fast 2D convolution. numpy and scipy libraries can be super efficient and might serve your needs better.</p>

python numpy multiprocessing