reduceByKey in spark for adding tuples
<p>Consider an Rdd with below dataset 
where 10000241 is the key and remaining are values</p>

<pre><code> ('10000241',([0,0,1],[None,None,'RX']))
 ('10000241',([0,2,0],[None,'RX','RX']))
 ('10000241',([3,0,0],['RX',None,None]))


pv1 = rdd.reduceBykey(lambda x,y :(
                   addtup(x[0],y[0]),
                   addtup(x[1],y[1]),
                ))


def addtup(t1,t2):
    j =()
    for k,v in enumerate(t1):
        j = j + (t1[k] + t2[k],)
     return j
</code></pre>

<p>The final output i want is (10000241,(3,2,1)('RX','RX','RX)) 
but i get the error of cant add none type to none type or nonetype to Str .how can i overcome this issue?</p>

<p>If I understood you correctly, you want to summarize numbers in the first tuple and to use logic or in the second?</p>

<p>I think you should rewrite your function as following:</p>

<pre><code>def addtup(t1,t2):
  left = list(map(lambda x: sum(x), zip(t1[0], t2[0])))
  right = list(map(lambda x: x[0] or x[1], zip(t1[1], t2[1])))
  return (left, right)
</code></pre>

<p>Then you can use it like this:</p>

<pre><code>rdd.reduceBykey(addtup)
</code></pre>

<p>Here is a demonstration</p>

<pre><code>import functools

data = (([0,0,1],[None,None,'RX']), 
 ([0,2,0],[None,'RX','RX']),
 ([3,0,0],['RX',None,None]))


functools.reduce(addtup, data)
#=&gt; ([3, 2, 1], ['RX', 'RX', 'RX'])
</code></pre>

python apache-spark pyspark apache-spark-sql