If something is persisted you can easily see them under the Storage tab in the web ui.
Thanks Best Regards On Tue, Nov 18, 2014 at 7:26 PM, Aniket Bhatnagar < aniket.bhatna...@gmail.com> wrote: > I am trying to figure out if sorting is persisted after applying Pair RDD > transformations and I am not able to decisively tell after reading the > documentation. > > For example: > val numbers = .. // RDD of numbers > val pairedNumbers = numbers.map(number => (number % 100, number)) > val sortedPairedNumbers = pairedNumbers.sortBy(pairedNumber => > pairedNumber._2) // Sort by values in the pair > val aggregates = sortedPairedNumbers.combineByKey(..) > > In this example, will the combine functions see values in sorted order? > What if I had done groupByKey and then combineByKey? What transformations > can unsort an already sorted data? >
