Re: merge 3 different types of RDDs in one

Hi Jacek, Thanks for the suggestion, i am going to try union. And what is your opinion on 2nd question. Thanks Shams On Tue, Dec 1, 2015 at 3:23 PM, Jacek Laskowski wrote: > Hi, > > Never done it before, but just yesterday I found out about > SparkContext.union method that

Re: merge 3 different types of RDDs in one

cogroup could be useful to you, since all three are PairRDD's. https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.rdd.PairRDDFunctions Best Regards, Praveen On 01.12.2015 10:47, Shams ul Haque wrote: Hi All, I have made 3 RDDs of 3 different dataset, all RDDs are

Re: merge 3 different types of RDDs in one

I think you should be able to join different rdds with same key. Have you tried that? On Dec 1, 2015 3:30 PM, "Praveen Chundi" wrote: > cogroup could be useful to you, since all three are PairRDD's. > > >

Re: merge 3 different types of RDDs in one

Hi, I have myself used union in a similar case. And applied reduceByKey on it. Union + reduceByKey will suffice join... but you will have to first use Map so that all values are of same datatype Regards, Sushrut Ikhar [image: https://]about.me/sushrutikhar

Re: merge 3 different types of RDDs in one

On Tue, Dec 1, 2015 at 10:57 AM, Shams ul Haque wrote: > Thanks for the suggestion, i am going to try union. ...and please report your findings back. > And what is your opinion on 2nd question. Dunno. If you find a solution, let us know. Jacek