Re: reverse an rdd

2014-10-16 Thread Sean Owen
Since you're concerned with the particular ordering, you will need to sort your RDD to ensure the ordering you have in mind. Simply reverse the Ordering with Ordering.reverse() and sort by that instead, and then use toLocalIterator() I suppose. Depending on what you're really trying to achieve,

Re: reverse an rdd

2014-10-16 Thread Paweł Szulc
Just to have this clear, can you answer with quick yes or no: Does it mean that when I create RDD from a file and I simply iterate through it like this: sc.textFile(some_text_file.txt).foreach(line = println(line)) then the actual lines might come in different order then they are in the file?

Re: reverse an rdd

2014-10-16 Thread Paweł Szulc
Nevermind, I've just run the code in the REPL. Indeed if we do not sort, then the order is totally random. Which actually makes sens if you think about it On Thu, Oct 16, 2014 at 9:58 PM, Paweł Szulc paul.sz...@gmail.com wrote: Just to have this clear, can you answer with quick yes or no: