Python multiprocessing silent failure with class
<p>the following does not work using python 2.7.9, but also does not throw any error or exception. is there a bug, or can multiprocessing not be used in a class?</p>

<pre><code>from multiprocessing import Pool

def testNonClass(arg):
    print "running %s" % arg
    return arg

def nonClassCallback(result):
    print "Got result %s" % result

class Foo:
    def __init__(self):
        po = Pool()
        for i in xrange(1, 3):
            po.apply_async(self.det, (i,), callback=self.cb)
        po.close()
        po.join()
        print "done with class"
    po = Pool()
    for i in xrange(1, 3):
        po.apply_async(testNonClass, (i,), callback=nonClassCallback)
    po.close()
    po.join()

    def cb(self, r):
        print "callback with %s" % r

    def det(self, M):
        print "method"
        return M+2


if __name__ == "__main__":
    Foo()
</code></pre>

<p>running prints this: </p>

<pre><code>done with class
running 1
running 2
Got result 1
Got result 2
</code></pre>

<p>EDIT: THis seems related, but it uses <code>.map</code>, while I specifically am needing to use <code>apply_async</code> which seems to matter in terms of how multiprocessing works with class instances (e.g. I dont have a picklnig error, like many other questions related to this) - <a href="http://stackoverflow.com/questions/29009790/python-how-to-do-multiprocessing-inside-of-a-class">Python how to do multiprocessing inside of a class?</a></p>

<p>Processes don't share state or memory by default, each process is an independent program. You need to either 1) use threading 2) use <a href="https://docs.python.org/2/library/multiprocessing.html#sharing-state-between-processes" rel="nofollow">specific types capable of sharing state</a> or 3) design your program to avoid shared state and rely on return values instead.</p>

<p>Update</p>

<p>You have two issues in your code, and one is masking the other.</p>

<p>1) You don't do anything with the result of the <code>apply_async</code>, I see that you're using callbacks, but you still need to catch the results and handle them. Because you're not doing this, you're not seeing the error caused by the second problem.</p>

<p>2) Methods of an object cannot be passed to other processes... I was really annoyed when I first discovered this, but there is an easy workaround. Try this:</p>

<pre><code>from multiprocessing import Pool

def _remote_det(foo, m):
    return foo.det(m)

class Foo:
    def __init__(self):
        pass
        po = Pool()
        results = []
        for i in xrange(1, 3):
            r = po.apply_async(_remote_det, (self, i,), callback=self.cb)
            results.append(r)

        po.close()
        for r in results:
            r.wait()
            if not r.successful():
                # Raises an error when not successful
                r.get()

        po.join()
        print "done with class"

    def cb(self, r):
        print "callback with %s" % r

    def det(self, M):
        print "method"
        return M+2

if __name__ == "__main__":
    Foo()
</code></pre>

python class multiprocessing