Re: Need help for RDD/DF transformation.

2017-03-30 Thread Yong Zhang
you can just pick the first element out from the Array "keys" of DF2, to join. Otherwise, I don't see any way to avoid a cartesian join. Yong ________ From: Mungeol Heo <mungeol@gmail.com> Sent: Thursday, March 30, 2017 3:05 AM To: ayan guha Cc: Yong Zhang

Re: Need help for RDD/DF transformation.

2017-03-30 Thread Mungeol Heo
7 at 9:03 PM, Yong Zhang <java8...@hotmail.com> wrote: >> > What is the desired result for >> > >> > >> > RDD/DF 1 >> > >> > 1, a >> > 3, c >> > 5, b >> > >> > RDD/DF 2 >> > >> > [1, 2, 3]

Re: Need help for RDD/DF transformation.

2017-03-30 Thread ayan guha
e desired result for > > > > > > RDD/DF 1 > > > > 1, a > > 3, c > > 5, b > > > > RDD/DF 2 > > > > [1, 2, 3] > > [4, 5] > > > > > > Yong > > > > > > From: Mungeol He

Re: Need help for RDD/DF transformation.

2017-03-29 Thread Mungeol Heo
com> wrote: > What is the desired result for > > > RDD/DF 1 > > 1, a > 3, c > 5, b > > RDD/DF 2 > > [1, 2, 3] > [4, 5] > > > Yong > > > From: Mungeol Heo <mungeol....@gmail.com> > Sent: Wednes

Re: Need help for RDD/DF transformation.

2017-03-29 Thread Yong Zhang
What is the desired result for RDD/DF 1 1, a 3, c 5, b RDD/DF 2 [1, 2, 3] [4, 5] Yong From: Mungeol Heo <mungeol@gmail.com> Sent: Wednesday, March 29, 2017 5:37 AM To: user@spark.apache.org Subject: Need help for RDD/DF transformation. Hello, S

Need help for RDD/DF transformation.

2017-03-29 Thread Mungeol Heo
Hello, Suppose, I have two RDD or data frame like addressed below. RDD/DF 1 1, a 3, a 5, b RDD/DF 2 [1, 2, 3] [4, 5] I need to create a new RDD/DF like below from RDD/DF 1 and 2. 1, a 2, a 3, a 4, b 5, b Is there an efficient way to do this? Any help will be great. Thank you.