Hi Sandy,
Sorry for the bothering.
The tests run ok even the SPARK_CLASS setting is there now, but It gives a
config warning and will potential interfere other settings like Marcelo said.
The warning goes away if I remove it out.
And Marcelo, I believe the setting in core/pom should not be used any more. But
I don't think it's worthy to file a JIRA for such small change. Maybe put it
into other related JIRA. It's a pity that your pr
already got merged.
--
Ye Xianjin
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Friday, September 26, 2014 at 6:29 AM, Sandy Ryza wrote:
> Hi Ye,
>
> I think git blame shows me because I fixed the formatting in core/pom.xml,
> but I don't actually know the original reason for setting SPARK_CLASSPATH
> there.
>
> Do the tests run OK if you take it out?
>
> -Sandy
>
>
> On Thu, Sep 25, 2014 at 1:59 AM, Ye Xianjin <[email protected]
> (mailto:[email protected])> wrote:
> > hi, Sandy Ryza:
> > I believe It's you originally added the SPARK_CLASSPATH in
> > core/pom.xml in the org.scalatest section. Does this still needed in 1.1?
> > I noticed this setting because when I looked into the unit-tests.log,
> > It shows something below:
> > > 14/09/24 23:57:19.246 WARN SparkConf:
> > > SPARK_CLASSPATH was detected (set to 'null').
> > > This is deprecated in Spark 1.0+.
> > >
> > > Please instead use:
> > > - ./spark-submit with --driver-class-path to augment the driver classpath
> > > - spark.executor.extraClassPath to augment the executor classpath
> > >
> > > 14/09/24 23:57:19.246 WARN SparkConf: Setting
> > > 'spark.executor.extraClassPath' to 'null' as a work-around.
> > > 14/09/24 23:57:19.247 WARN SparkConf: Setting
> > > 'spark.driver.extraClassPath' to 'null' as a work-around.
> >
> > However I didn't set SPARK_CLASSPATH env variable. And looked into the
> > SparkConf.scala, If user actually set extraClassPath, the SparkConf will
> > throw SparkException.
> > --
> > Ye Xianjin
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >
> >
> > On Tuesday, September 23, 2014 at 12:56 AM, Ye Xianjin wrote:
> >
> > > Hi:
> > > I notice the scalatest-maven-plugin set SPARK_CLASSPATH environment
> > > variable for testing. But in the SparkConf.scala, this is deprecated in
> > > Spark 1.0+.
> > > So what this variable for? should we just remove this variable?
> > >
> > >
> > > --
> > > Ye Xianjin
> > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> > >
> >
>
