partition your input into even number partitions use mapPartition to operate on Iterator[Int]
maybe there are some more efficient way…. Best, -- Nan Zhu On Monday, March 24, 2014 at 7:59 PM, yh18190 wrote: > Hi, I have large data set of numbers ie RDD and wanted to perform a > computation only on groupof two values at a time. For example > 1,2,3,4,5,6,7... is an RDD Can i group the RDD into (1,2),(3,4),(5,6)...?? > and perform the respective computations ?in an efficient manner? As we do'nt > have a way to index elements directly using forloop etc..(i,i+1)...is their > way to resolve this problem? Please suggest me ..i would be really thankful > to you > View this message in context: Splitting RDD and Grouping together to perform > computation > (http://apache-spark-user-list.1001560.n3.nabble.com/Splitting-RDD-and-Grouping-together-to-perform-computation-tp3153.html) > Sent from the Apache Spark User List mailing list archive > (http://apache-spark-user-list.1001560.n3.nabble.com/) at Nabble.com > (http://Nabble.com).
