Re: Debug what is replication Level of which RDD

How many RDDs are you persisting? If its 2, then you can verify it by disabling the persist for one of them and from the UI you can see which one of mappedRDD/shuffledRDD. Thanks Best Regards On Sun, Jan 24, 2016 at 3:25 AM, gaurav sharma wrote: > Hi All, > > I have

Debug what is replication Level of which RDD

Hi All, I have enabled replication for my RDDs. I see on the Storage tab of the Spark UI, which mentions the replication level 2x or 1x. But the names given are mappedRDD, shuffledRDD, I am not able to debug which of my RDD is 2n replicated, and which one is 1x. Please help. Regards, Gaurab