I think it is one of the conceptual difference in Spark compare to other languages, there is no indexing in plain RDDs, This was the thread with Ankit:
Yes. So order preservation can not be guaranteed in the case of failure. Also not sure if partitions are ordered. Can you get the same sequence of partitions in mapPartition? On 13 Sep 2017 19:54, "Ankit Maloo" <[email protected]> wrote: > > Rdd are fault tolerant as it can be recomputed using DAG without storing the > intermediate RDDs. > > On 13-Sep-2017 11:16 PM, "Suzen, Mehmet" <[email protected]> wrote: >> >> But what happens if one of the partitions fail, how fault tolerance recover >> elements in other partitions. --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
