Re: How to subtract two RDDs with same size

Hi Zhiliang, How about doing something like this? val rdd3 = rdd1.zip(rdd2).map(p => p._1.zip(p._2).map(z => z._1 - z._2)) The first zip will join the two RDDs and produce an RDD of (Array[Float], Array[Float]) pairs. On each pair, we zip the two Array[Float] components together to form an

How to subtract two RDDs with same size

Hi All, There are two RDDs :  RDD rdd1, and RDD rdd2,that is to say, rdd1 and rdd2 are similar with DataFrame, or Matrix with same row number and column number. I would like to get RDD rdd3,  each element in rdd3 is the subtract between rdd1 and rdd2 of thesame position,

Re: How to subtract two RDDs with same size

there is matrix add API, might map rdd2 each row element to be negative , then make rdd1 and rdd2 and call add ? Or some more ways ... On Wednesday, September 23, 2015 3:11 PM, Zhiliang Zhu wrote: Hi All, There are two RDDs :  RDD rdd1, and

Re: How to subtract two RDDs with same size

Hi Sujit, It is wonderful for you!I must show my sincere appreciation towards your kind help. Thank you very much!Best Regards,Zhiliang        On Wednesday, September 23, 2015 10:15 PM, Sujit Pal