Fail to map the output of a generator to a function in multiprocessing with python-2.7
<p>Here is my code:</p>

<pre><code>from multiprocessing import Pool

user_list = [1, 2, 3, 4, 5]
def gen_pair():
    for u1 in reversed(user_list):
        for u2 in reversed(list(range(1, u1))):
            yield (u1, u2)

def cal_sim(u_pair):
     u1, u2 = u_pair
     sim = sim_f(df[u1], df[u2])
     return sim

pool = Pool(processes=6)
vals = pool.map(cal_sim, gen_pair())
df2record = pd.DataFrame(columns=['u1', 'u2', 'js'])
for v in vals:
    print (v)
pool.terminate()
</code></pre>

<p>But when I run the code, I got such an error: <code>TypeError: unsupported operand type(s) for +: 'set' and 'set'</code>. The full TraceBack is as below:</p>

<pre><code>Traceback (most recent call last):
  File "b.py", line 57, in &lt;module&gt;
    main()
  File "b.py", line 47, in main
    vals = pool.map(cal_sim, gen_pair())
  File "yobichi/python/2.7.10_2/lib/python2.7/multiprocessing/pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "yobichi/python/2.7.10_2/lib/python2.7/multiprocessing/pool.py", line 567, in get
    raise self._value
TypeError: unsupported operand type(s) for +: 'set' and 'set'
</code></pre>

<p>Could you please tell me what is the reason and how can I deal with it appropriately? </p>

<p>You have an infinite recursion here:</p>

<pre><code>def cal_sim(u_pair):
    u1, u2 = u_pair
    sim = cal_sim(df[u1], df[u2])
    return sim
</code></pre>

<p>Because <code>cal_sim</code> calls <code>cal_sim</code> without any end condition.</p>

<p>There is also a problem with the parameters, because <code>cal_sim</code> has 1 parameter and you call it with 2 arguments.</p>

python python-2.7 multiprocessing