You mark these provided, which is correct. If the version of Scala provided at runtime differs, you'll have a problem.
In fact you can also see you mixed Scala versions in your dependencies here. MLlib is on 2.10. On Fri, Jul 22, 2016 at 6:49 PM, VG <vlin...@gmail.com> wrote: > Sean, > > I am only using the maven dependencies for spark in my pom file. > I don't have anything else. I guess maven dependency should resolve to the > correct scala version .. isn;t it ? Any ideas. > > <dependency><!-- Spark --> > <groupId>org.apache.spark</groupId> > <artifactId>spark-core_2.11</artifactId> > <version>2.0.0-preview</version> > <scope>provided</scope> > </dependency> > <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 --> > <dependency><!-- Spark SQL --> > <groupId>org.apache.spark</groupId> > <artifactId>spark-sql_2.11</artifactId> > <version>2.0.0-preview</version> > <scope>provided</scope> > </dependency> > <!-- > https://mvnrepository.com/artifact/org.apache.spark/spark-streaming_2.11 --> > <dependency><!-- Spark Streaming --> > <groupId>org.apache.spark</groupId> > <artifactId>spark-streaming_2.11</artifactId> > <version>2.0.0-preview</version> > <scope>provided</scope> > </dependency> > <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10 > --> > <dependency> > <groupId>org.apache.spark</groupId> > <artifactId>spark-mllib_2.10</artifactId> > <version>2.0.0-preview</version> > <scope>provided</scope> > </dependency> > > > > On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen <so...@cloudera.com> wrote: >> >> -dev >> Looks like you are mismatching the version of Spark you deploy on at >> runtime then. Sounds like it was built for Scala 2.10 >> >> On Fri, Jul 22, 2016 at 6:43 PM, VG <vlin...@gmail.com> wrote: >> > Using 2.0.0-preview using maven >> > So all dependencies should be correct I guess >> > >> > <dependency><!-- Spark --> >> > <groupId>org.apache.spark</groupId> >> > <artifactId>spark-core_2.11</artifactId> >> > <version>2.0.0-preview</version> >> > <scope>provided</scope> >> > </dependency> >> > >> > I see in maven dependencies that this brings in >> > scala-reflect-2.11.4 >> > scala-compiler-2.11.0 >> > >> > and so on >> > >> > >> > >> > On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ailov...@wayfair.com> >> > wrote: >> >> >> >> What version of Spark/Scala are you running? >> >> >> >> >> >> >> >> -Aaron >> > >> > > > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
