Hi Yang, It looks like your build file is a different version than the version of Spark you are running against. I'd try building against the same version of spark as you are running your application against (1.1.0). Also what is your assembly/shading configuration for your build?
Cheers, Holden :) On Wed, Oct 22, 2014 at 5:39 PM, Yang <teddyyyy...@gmail.com> wrote: > > I'm trying to submit a simple test code through spark-submit. first > portion of the code works fine, but some calls to breeze vector library > fails: > > 14/10/22 17:36:02 INFO CacheManager: Partition rdd_1_0 not found, > computing it > 14/10/22 17:36:02 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID > 0) > java.lang.NoSuchMethodError: > breeze.linalg.SparseVector$mcD$sp.<init>([I[DILbreeze/storage/DefaultArrayValue;)V > at spark.kmeans.SparseVector.toBreeze(Vectors.scala:182) > at spark.kmeans.MyTest$$anonfun$1.apply(MyTest.scala:56) > at spark.kmeans.MyTest$$anonfun$1.apply(MyTest.scala:56) > at scala.collection.Iterator$$anon$11.next(Iterator.scala:328) > at org.apache.spark.storage.MemoryStore.unrollSafely(MemoryStore.scala:236) > > > I ran this code with > ~/tools/spark-1.1.0-bin-hadoop2.4/bin/spark-submit --class > spark.kmeans.MyTest --master local target/reco-v1-jar-with-dependencies.jar > > > > my system scala version is 2.10.4 > > the relevant sections in my pom is: (I did not build with sbt, since it > gave me some problems) > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-core_2.10</artifactId> > <version>1.0.2</version> > <exclusions> > <exclusion> > > <groupId>org.apache.hadoop</groupId> > > <artifactId>hadoop-client</artifactId> > </exclusion> > </exclusions> > </dependency> > > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-mllib_2.10</artifactId> > <version>1.0.2</version> > </dependency> > > <dependency> > <groupId>org.scala-lang</groupId> > <artifactId>scala-library</artifactId> > <version>2.10.4</version> > </dependency> > > > > Thanks a lot > Yang > -- Cell : 425-233-8271
