Hi Ayan,
Thanks for the reply. Yes that is what i am currently doing. I thought
there may be a more efficient way provided by spark that i could use
directly.
Best Regards,
Pulasthi
On Thu, May 19, 2016 at 6:42 PM, ayan guha wrote:
> You can add the index from mappartitionwithindex in the outp
You can add the index from mappartitionwithindex in the output and order
based on that in merge step
On 19 May 2016 13:22, "Pulasthi Supun Wickramasinghe"
wrote:
> Hi Devs/All,
>
> I am pretty new to Spark. I have a program which does some map reduce
> operations with matrices. Here *shortrddFina
Hi Devs/All,
I am pretty new to Spark. I have a program which does some map reduce
operations with matrices. Here *shortrddFinal* is a of type "
*RDD[Array[Short]]"* and consists of several partitions
*var BC =
shortrddFinal.mapPartitionsWithIndex(calculateBCInternal).reduce(mergeBC)*
The map fu
