It's likely that with codegen, you need a bigger permgen space. Also I found that Java 7 doesn't do very well w.r.t. flushing code cache. As a result, Spark SQL and DataFrames now run much better under Java 8, because it flushes code cache better.
On Wed, Sep 9, 2015 at 2:12 PM, Sandy Ryza <sandy.r...@cloudera.com> wrote: > Java 7. > > FWIW I was just able to get it to work by increasing MaxPermSize to 256m. > > -Sandy > > On Wed, Sep 9, 2015 at 11:37 AM, Reynold Xin <r...@databricks.com> wrote: > >> Java 7 / 8? >> >> On Wed, Sep 9, 2015 at 10:10 AM, Sandy Ryza <sandy.r...@cloudera.com> >> wrote: >> >>> I just upgraded the spark-timeseries >>> <https://github.com/cloudera/spark-timeseries> project to run on top of >>> 1.5, and I'm noticing that tests are failing with OOMEs. >>> >>> I ran a jmap -histo on the process and discovered the top heap items to >>> be: >>> 1: 163428 22236064 <methodKlass> >>> 2: 163428 21112648 <constMethodKlass> >>> 3: 12638 14459192 <constantPoolKlass> >>> 4: 12638 13455904 <instanceKlassKlass> >>> 5: 10539 7642528 <constantPoolCacheKlass> >>> >>> Not sure whether this is suspicious. Any ideas? >>> >>> -Sandy >>> >> >> >
