RE: Driver OOM after upgrading to 1.5

Will that be helpful if adding jvm options like: -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled From: Reynold Xin [mailto:r...@databricks.com] Sent: Thursday, September 10, 2015 5:31 AM To: Sandy Ryza Cc: user@spark.apache.org Subject: Re: Driver OOM after upgrading to 1.5 It's

Re: Driver OOM after upgrading to 1.5

Java 7 / 8? On Wed, Sep 9, 2015 at 10:10 AM, Sandy Ryza wrote: > I just upgraded the spark-timeseries > project to run on top of > 1.5, and I'm noticing that tests are failing with OOMEs. > > I ran a jmap -histo on the

Driver OOM after upgrading to 1.5

I just upgraded the spark-timeseries project to run on top of 1.5, and I'm noticing that tests are failing with OOMEs. I ran a jmap -histo on the process and discovered the top heap items to be: 1:163428 22236064 2:

Re: Driver OOM after upgrading to 1.5

Java 7. FWIW I was just able to get it to work by increasing MaxPermSize to 256m. -Sandy On Wed, Sep 9, 2015 at 11:37 AM, Reynold Xin wrote: > Java 7 / 8? > > On Wed, Sep 9, 2015 at 10:10 AM, Sandy Ryza > wrote: > >> I just upgraded the

Re: Driver OOM after upgrading to 1.5

It's likely that with codegen, you need a bigger permgen space. Also I found that Java 7 doesn't do very well w.r.t. flushing code cache. As a result, Spark SQL and DataFrames now run much better under Java 8, because it flushes code cache better. On Wed, Sep 9, 2015 at 2:12 PM, Sandy Ryza