Hi all,
I'm using some functions from Breeze in a spark job but I get the following
build error :
*Error:scalac: bad symbolic reference. A signature in RandBasis.class
refers to term math3*
*in package org.apache.commons which is not available.*
*It may be completely missing from the current
spark-core's dependency on commons-math3 is @ test scope (core/pom.xml):
dependency
groupIdorg.apache.commons/groupId
artifactIdcommons-math3/artifactId
version3.3/version
scopetest/scope
/dependency
Adjusting the scope should solve the problem below.
On Fri, Sep
Thank Ted. Can you tell me how to adjust the scope ?
On Fri, Sep 26, 2014 at 5:47 PM, Ted Yu yuzhih...@gmail.com wrote:
spark-core's dependency on commons-math3 is @ test scope (core/pom.xml):
dependency
groupIdorg.apache.commons/groupId
artifactIdcommons-math3/artifactId
Shouldn't the user's application depend on commons-math3 if it uses
it? it shouldn't require a Spark change. Maybe I misunderstand.
On Fri, Sep 26, 2014 at 4:47 PM, Ted Yu yuzhih...@gmail.com wrote:
spark-core's dependency on commons-math3 is @ test scope (core/pom.xml):
dependency
You can use scope of runtime.
See
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
Cheers
On Fri, Sep 26, 2014 at 8:57 AM, Jaonary Rabarisoa jaon...@gmail.com
wrote:
Thank Ted. Can you tell me how to adjust the scope ?
On Fri, Sep 26,
I solve the problem by including the commons-math3 package in my sbt
dependencies as Sean suggested. Thanks.
On Fri, Sep 26, 2014 at 6:05 PM, Ted Yu yuzhih...@gmail.com wrote:
You can use scope of runtime.
See
We removed commons-math3 from dependencies to avoid version conflict
with hadoop-common. hadoop-common-2.3+ depends on commons-math3-3.1.1,
while breeze depends on commons-math3-3.3. 3.3 is not backward
compatible with 3.1.1. So we removed it because the breeze functions
we use do not touch
