We need to create RDDas below JavaPairRDD<String,List<HashMap<String,List<String>>>>
The idea is we need to do lookup() on Key which will return a list of hash maps kind of structure and then do lookup on subkey which is the key in the HashMap returned _____________________________ From: Silas Davis <[email protected]<mailto:[email protected]>> Sent: Wednesday, August 19, 2015 10:34 pm Subject: Re: Creating RDD with key and Subkey To: Ratika Prasad <[email protected]<mailto:[email protected]>>, <[email protected]<mailto:[email protected]>> This should be sent to the user mailing list, I think. It depends what you want to do with the RDD, so yes you could throw around (String, HashMap<String,List<String>>) tuples or perhaps you'd like to be able to groupByKey, reduceByKey on the key and sub-key as a composite in which case JavaPairRDD<Tuple2<String,String>, List<String>> might be more appropriate. Not really clear what you are asking. On Wed, 19 Aug 2015 at 17:15 Ratika Prasad < [email protected]<mailto:[email protected]>> wrote: Hi, We have a need where we need the RDD with the following format JavaPairRDD<String,HashMap<String,List<String>>>, mostly RDD with a Key and Subkey kind of a structure, how is that doable in Spark ? Thanks R
