Re: special case of custom partitioning

2014-03-07 Thread Manoj Awasthi
Thanks Mayur - based on the doc-comments in source looks like this will work for the case. I will confirm. the dreamers of the day are dangerous men, for they may act their dream with open eyes, and make it possible On Fri, Mar 7, 2014 at 2:21 AM, Mayur Rustagi

special case of custom partitioning

2014-03-06 Thread Manoj Awasthi
Hi All, I have a three machine cluster. I have two RDDs each consisting of (K,V) pairs. RDDs have just three keys 'a', 'b' and 'c'. // list1 - List(('a',1), ('b',2), val rdd1 = sc.parallelize(list1).groupByKey(new HashPartitioner(3)) // list2 - List(('a',2), ('b',7),