Great. thanks a ton for helping out on this Sean. I somehow messed this up (and was running in loops for last 2 hours )
thanks again -VG On Fri, Jul 22, 2016 at 11:28 PM, Sean Owen <so...@cloudera.com> wrote: > You mark these provided, which is correct. If the version of Scala > provided at runtime differs, you'll have a problem. > > In fact you can also see you mixed Scala versions in your dependencies > here. MLlib is on 2.10. > > On Fri, Jul 22, 2016 at 6:49 PM, VG <vlin...@gmail.com> wrote: > > Sean, > > > > I am only using the maven dependencies for spark in my pom file. > > I don't have anything else. I guess maven dependency should resolve to > the > > correct scala version .. isn;t it ? Any ideas. > > > > <dependency><!-- Spark --> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-core_2.11</artifactId> > > <version>2.0.0-preview</version> > > <scope>provided</scope> > > </dependency> > > <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 > --> > > <dependency><!-- Spark SQL --> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-sql_2.11</artifactId> > > <version>2.0.0-preview</version> > > <scope>provided</scope> > > </dependency> > > <!-- > > https://mvnrepository.com/artifact/org.apache.spark/spark-streaming_2.11 > --> > > <dependency><!-- Spark Streaming --> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-streaming_2.11</artifactId> > > <version>2.0.0-preview</version> > > <scope>provided</scope> > > </dependency> > > <!-- > https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10 > > --> > > <dependency> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-mllib_2.10</artifactId> > > <version>2.0.0-preview</version> > > <scope>provided</scope> > > </dependency> > > > > > > > > On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen <so...@cloudera.com> wrote: > >> > >> -dev > >> Looks like you are mismatching the version of Spark you deploy on at > >> runtime then. Sounds like it was built for Scala 2.10 > >> > >> On Fri, Jul 22, 2016 at 6:43 PM, VG <vlin...@gmail.com> wrote: > >> > Using 2.0.0-preview using maven > >> > So all dependencies should be correct I guess > >> > > >> > <dependency><!-- Spark --> > >> > <groupId>org.apache.spark</groupId> > >> > <artifactId>spark-core_2.11</artifactId> > >> > <version>2.0.0-preview</version> > >> > <scope>provided</scope> > >> > </dependency> > >> > > >> > I see in maven dependencies that this brings in > >> > scala-reflect-2.11.4 > >> > scala-compiler-2.11.0 > >> > > >> > and so on > >> > > >> > > >> > > >> > On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ailov...@wayfair.com > > > >> > wrote: > >> >> > >> >> What version of Spark/Scala are you running? > >> >> > >> >> > >> >> > >> >> -Aaron > >> > > >> > > > > > >
