Re: Slower performance when bigger memory?

2015-04-24 Thread Shawn Zheng
this is not about gc issue itself. The memory is On Friday, April 24, 2015, Evo Eftimov evo.efti...@isecc.com wrote: You can resort to Serialized storage (still in memory) of your RDDs – this will obviate the need for GC since the RDD elements are stored as serialized objects off the JVM heap

Re: Process time series RDD after sortByKey

2015-03-17 Thread Shawn Zheng
');] *Sent:* Monday, March 16, 2015 11:22 AM *To:* Shawn Zheng; user@spark.apache.org javascript:_e(%7B%7D,'cvml','user@spark.apache.org'); *Subject:* Re: Process time series RDD after sortByKey Hi Shuai, On Sat, Mar 14, 2015 at 11:02 AM, Shawn Zheng szheng.c...@gmail.com javascript:_e

Re: jar conflict with Spark default packaging

2015-03-16 Thread Shawn Zheng
Thanks a lot. I will give a try! On Monday, March 16, 2015, Adam Lewandowski adam.lewandow...@gmail.com wrote: Prior to 1.3.0, Spark has 'spark.files.userClassPathFirst' for non-yarn apps. For 1.3.0, use 'spark.executor.userClassPathFirst'. See