I'm trying to submit a simple test code through spark-submit. first portion of the code works fine, but some calls to breeze vector library fails:
14/10/22 17:36:02 INFO CacheManager: Partition rdd_1_0 not found, computing it 14/10/22 17:36:02 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0) java.lang.NoSuchMethodError: breeze.linalg.SparseVector$mcD$sp.<init>([I[DILbreeze/storage/DefaultArrayValue;)V at spark.kmeans.SparseVector.toBreeze(Vectors.scala:182) at spark.kmeans.MyTest$$anonfun$1.apply(MyTest.scala:56) at spark.kmeans.MyTest$$anonfun$1.apply(MyTest.scala:56) at scala.collection.Iterator$$anon$11.next(Iterator.scala:328) at org.apache.spark.storage.MemoryStore.unrollSafely(MemoryStore.scala:236) I ran this code with ~/tools/spark-1.1.0-bin-hadoop2.4/bin/spark-submit --class spark.kmeans.MyTest --master local target/reco-v1-jar-with-dependencies.jar my system scala version is 2.10.4 the relevant sections in my pom is: (I did not build with sbt, since it gave me some problems) <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.10</artifactId> <version>1.0.2</version> <exclusions> <exclusion> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-client</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-mllib_2.10</artifactId> <version>1.0.2</version> </dependency> <dependency> <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>2.10.4</version> </dependency> Thanks a lot Yang