Re: OFF_HEAP storage level

2014-07-04 Thread Ajay Srivastava
Thanks Jerry. It looks like a good option, will try it. Regards, Ajay On Friday, July 4, 2014 2:18 PM, "Shao, Saisai" wrote: Hi Ajay,   StorageLevel OFF_HEAP means for can cache your RDD into Tachyon, the prerequisite is that you should deploy Tachyon among Spark.   Yes, it can alleviate

RE: OFF_HEAP storage level

2014-07-04 Thread Shao, Saisai
Hi Ajay, StorageLevel OFF_HEAP means for can cache your RDD into Tachyon, the prerequisite is that you should deploy Tachyon among Spark. Yes, it can alleviate GC, since you offload JVM memory into system managed memory. You can use rdd.persist(...) to use this level, details can be checked in