Thats expected. I think sortByKey is option too & probably a better one.
Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Thu, Mar 20, 2014 at 3:20 PM, Ameet Kini <ameetk...@gmail.com> wrote: > > val rdd2 = rdd.partitionBy(my partitioner).reduceByKey(some function) > > I see that rdd2's partitions are not internally sorted. Can someone > confirm that this is expected behavior? And if so, the only way to get > partitions internally sorted is to follow it with something like this > > val rdd2 = rdd.partitionBy(my partitioner).reduceByKey(some > function).mapPartitions(p => sort(p)) > > Thanks, > Ameet > >
