Will that be helpful if adding jvm options like: -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled
From: Reynold Xin [mailto:r...@databricks.com] Sent: Thursday, September 10, 2015 5:31 AM To: Sandy Ryza Cc: user@spark.apache.org Subject: Re: Driver OOM after upgrading to 1.5 It's likely that with codegen, you need a bigger permgen space. Also I found that Java 7 doesn't do very well w.r.t. flushing code cache. As a result, Spark SQL and DataFrames now run much better under Java 8, because it flushes code cache better. On Wed, Sep 9, 2015 at 2:12 PM, Sandy Ryza <sandy.r...@cloudera.com<mailto:sandy.r...@cloudera.com>> wrote: Java 7. FWIW I was just able to get it to work by increasing MaxPermSize to 256m. -Sandy On Wed, Sep 9, 2015 at 11:37 AM, Reynold Xin <r...@databricks.com<mailto:r...@databricks.com>> wrote: Java 7 / 8? On Wed, Sep 9, 2015 at 10:10 AM, Sandy Ryza <sandy.r...@cloudera.com<mailto:sandy.r...@cloudera.com>> wrote: I just upgraded the spark-timeseries<https://github.com/cloudera/spark-timeseries> project to run on top of 1.5, and I'm noticing that tests are failing with OOMEs. I ran a jmap -histo on the process and discovered the top heap items to be: 1: 163428 22236064 <methodKlass> 2: 163428 21112648 <constMethodKlass> 3: 12638 14459192 <constantPoolKlass> 4: 12638 13455904 <instanceKlassKlass> 5: 10539 7642528 <constantPoolCacheKlass> Not sure whether this is suspicious. Any ideas? -Sandy
