Just provide own partitioner. One I wrote a partitioner which keeps similar keys together in one partitioner.
Best regards, Denis On 12 September 2016 at 19:44, sujeet jog <sujeet....@gmail.com> wrote: > Hi, > > Is there a way to partition set of data with n keys into exactly n > partitions. > > For ex : - > > tuple of 1008 rows with key as x > tuple of 1008 rows with key as y and so on total 10 keys ( x, y etc ) > > Total records = 10080 > NumOfKeys = 10 > > i want to partition the 10080 elements into exactly 10 partitions with > each partition having elements with unique key > > Is there a way to make this happen ?.. any ideas on implementing custom > partitioner. > > > The current partitioner i'm using is HashPartitioner from which there are > cases where key.hascode() % numPartitions for keys of x & y become same. > > hence many elements with different keys fall into single partition at > times. > > > > Thanks, > Sujeet > -- //with Best Regards --Denis Bolshakov e-mail: bolshakov.de...@gmail.com
