Re: Spark 2.0.0 performance; potential large Spark core regression

MGB Cc: Michael Allman <mich...@videoamp.com>, dev <dev@spark.apache.org> Date: 08/07/2016 17:26 Subject: Re: Spark 2.0.0 performance; potential large Spark core regression bq. we turned it off when fixing a bug Adam: Can you refer to the bug JIRA ? Thanks On Fri, Jul 8, 201

Re: Spark 2.0.0 performance; potential large Spark core regression

e can post the 1.6.2 comparison early next week, running lots of iterations over the weekend once we get the dedicated time again Cheers, From:Michael Allman <mich...@videoamp.com> To: Adam Roberts/UK/IBM@IBMGB Cc:dev <dev@spark.apache.org> Date:

Re: Spark 2.0.0 performance; potential large Spark core regression

dedicated time again >> >> Cheers, >> >> >> >> >> >> From:Michael Allman <mich...@videoamp.com >> <mailto:mich...@videoamp.com>> >> To:Adam Roberts/UK/IBM@IBMGB >> Cc:dev <dev@spark.apach

Re: Spark 2.0.0 performance; potential large Spark core regression

n > > Cheers, > > > > > > From:Michael Allman <mich...@videoamp.com> > To:Adam Roberts/UK/IBM@IBMGB > Cc: dev <dev@spark.apache.org> > Date:08/07/2016 16:44 > Subject:Re: Spark 2.0.0 performance; pot

Re: Spark 2.0.0 performance; potential large Spark core regression

MGB Cc: dev <dev@spark.apache.org> Date: 08/07/2016 16:44 Subject: Re: Spark 2.0.0 performance; potential large Spark core regression Hi Adam, Do you have your spark confs and your spark-env.sh somewhere where we can see them? If not, can you make them available? Cheers, Micha

Re: Spark 2.0.0 performance; potential large Spark core regression

Hi Adam, Do you have your spark confs and your spark-env.sh somewhere where we can see them? If not, can you make them available? Cheers, Michael > On Jul 8, 2016, at 3:17 AM, Adam Roberts wrote: > > Hi, we've been testing the performance of Spark 2.0 compared to

Spark 2.0.0 performance; potential large Spark core regression

Hi, we've been testing the performance of Spark 2.0 compared to previous releases, unfortunately there are no Spark 2.0 compatible versions of HiBench and SparkPerf apart from those I'm working on (see https://github.com/databricks/spark-perf/issues/108) With the Spark 2.0 version of SparkPerf