Re: Question about MEOMORY_AND_DISK persistence

Thank you Ashwin. On Sun, Feb 28, 2016 at 7:19 PM, Ashwin Giridharan wrote: > Hi Vishnu, > > A partition will either be in memory or in disk. > > -Ashwin > On Feb 28, 2016 15:09, "Vishnu Viswanath" > wrote: > >> Hi All, >> >> I have a

Re: Question about MEOMORY_AND_DISK persistence

Hi Vishnu, A partition will either be in memory or in disk. -Ashwin On Feb 28, 2016 15:09, "Vishnu Viswanath" wrote: > Hi All, > > I have a question regarding Persistence (MEMORY_AND_DISK) > > Suppose I am trying to persist an RDD which has 2 partitions and only 1

Question about MEOMORY_AND_DISK persistence

Hi All, I have a question regarding Persistence (MEMORY_AND_DISK) Suppose I am trying to persist an RDD which has 2 partitions and only 1 partition can be fit in memory completely but some part of partition 2 can also be fit, will spark keep the portion of partition 2 in memory and rest in disk,