For the record, the solution I was suggesting was about like this:
inputRDD.flatMap { input =
val tokens = input.split(',')
val id = tokens(0)
val keyValuePairs = tokens.tail.grouped(2)
val keys = keyValuePairs.map(_(0))
keys.map(key = (id, key))
}
This is much more efficient.
On Wed,
cool let me adapt that. thanks a tonregardssanjay
From: Sean Owen so...@cloudera.com
To: Sanjay Subramanian sanjaysubraman...@yahoo.com
Cc: user@spark.apache.org user@spark.apache.org
Sent: Monday, January 5, 2015 3:19 AM
Subject: Re: FlatMapValues
For the record, the solution I
PM
Subject: Re: FlatMapValues
thanks let me try that out
From: Hitesh Khamesra hiteshk...@gmail.com
To: Sanjay Subramanian sanjaysubraman...@yahoo.com
Cc: Kapil Malik kma...@adobe.com; Sean Owen so...@cloudera.com;
user@spark.apache.org user@spark.apache.org
Sent: Thursday, January
both can be used.
Regards,
Kapil
-Original Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: 31 December 2014 21:16
To: Sanjay Subramanian
Cc: user@spark.apache.org
Subject: Re: FlatMapValues
From the clarification below, the problem is that you are calling
From: Kapil Malik kma...@adobe.com
To: Sean Owen so...@cloudera.com; Sanjay Subramanian
sanjaysubraman...@yahoo.com
Cc: user@spark.apache.org user@spark.apache.org
Sent: Wednesday, December 31, 2014 9:35 AM
Subject: RE: FlatMapValues
Hi Sanjay,
Oh yes .. on flatMapValues, it's
Why don't you push \n instead of \t in your first transformation [
(fields(0),(fields(1)+\t+fields(3)+\t+fields(5)+\t+fields(7)+\t
+fields(9)))] and then do saveAsTextFile?
-Raghavendra
On Wed Dec 31 2014 at 1:42:55 PM Sanjay Subramanian
sanjaysubraman...@yahoo.com.invalid wrote:
hey guys
My
Hi Sanjay,
I tried running your code on spark shell piece by piece –
// Setup
val line1 = “025126,Chills,8.10,Injection site oedema,8.10,Injection site
reaction,8.10,Malaise,8.10,Myalgia,8.10”
val line2 = “025127,Chills,8.10,Injection site oedema,8.10,Injection site
Hi Sanjay,
Doing an if inside a Map sounds like a bad idea, it seems like you actually
want to filter and then apply map
On Wed, Dec 31, 2014 at 9:54 AM, Kapil Malik kma...@adobe.com wrote:
Hi Sanjay,
I tried running your code on spark shell piece by piece –
// Setup
val line1 =
to
1,red1,blue1,green2,yellow2,violet2,pink
thanks
regards
sanjay
From: Fernando O. fot...@gmail.com
To: Kapil Malik kma...@adobe.com
Cc: Sanjay Subramanian sanjaysubraman...@yahoo.com; user@spark.apache.org
user@spark.apache.org
Sent: Wednesday, December 31, 2014 6:06 AM
Subject: Re
be used.
Regards,
Kapil
-Original Message-
From: Sean Owen [mailto:so...@cloudera.com]
Sent: 31 December 2014 21:16
To: Sanjay Subramanian
Cc: user@spark.apache.org
Subject: Re: FlatMapValues
From the clarification below, the problem is that you are calling
flatMapValues, which
10 matches
Mail list logo
