Thank you very much for your reply. As I mentioned earlier, I am using mahoutSparkContext, and MAHOUT_HOME is set to the correct mahout path. I also have tried setting up the context myself as I looked into the implementation of mahoutSparkContext and supplied the jars path manually. still the same error. I will try with spark 1.5 and report.
Thank you very much again, Kind Regards, Bahaa On Tue, Feb 2, 2016 at 12:01 PM, Dmitriy Lyubimov <dlie...@gmail.com> wrote: > Bahaa, first off, i don't think we have certified any of releases to run > with spar 1.6 (yet). I think spark 1.5 is the last known release to run > with 0.11 series. > > Second, if you use mahoutSparkContext() method to create context, it would > look for MAHOUT_HOME setup to add mahout binaries to the job. So the > reasons you may not getting it is perhaps you are not using the > mahoutCreateContext()? > > alternatively, you can create context yourself, but you need (1) make sure > it has enabled and configured Kryo serialization properly, and (2) have > added all necessary mahout jars on your own. > > -d > > On Tue, Feb 2, 2016 at 8:22 AM, BahaaEddin AlAila <bahaelai...@gmail.com> > wrote: > > > Greetings mahout users, > > > > I have been trying to use mahout samsara as a library with scala/spark, > but > > I haven't been successful in doing so. > > > > I am running spark 1.6.0 binaries, didn't build it myself. > > However, I tried both readily available binaries on Apache mirrors, and > > cloning and compiling mahout's repo, but neither worked. > > > > I keep getting > > > > Exception in thread "main" java.lang.NoClassDefFoundError: > > org/apache/mahout/sparkbindings/SparkDistributedContext > > > > The way I am doing things is: > > I have spark in ~/spark-1.6 > > and mahout in ~/mahout > > I have set both $SPARK_HOME and $MAHOUT_HOME accordingly, along with > > $MAHOUT_LOCAL=true > > > > and I have: > > > > ~/app1/build.sbt > > ~/app1/src/main/scala/App1.scala > > > > in build.sbt I have these lines to declare mahout dependecies: > > > > libraryDependencies += "org.apache.mahout" %% "mahout-math-scala" % > > "0.11.1" > > > > libraryDependencies += "org.apache.mahout" % "mahout-math" % "0.11.1" > > > > libraryDependencies += "org.apache.mahout" % "mahout-spark_2.10" % > "0.11.1" > > > > along with other spark dependencies > > > > and in App1.scala, in the main function, I construct a context object > using > > mahoutSparkContext, and of course, the sparkbindings are imported > > > > everything compiles successfully > > > > however, when I submit to spark, I get the above mentioned error. > > > > I have a general idea of why this is happening: because the compiled app1 > > jar depends on mahout-spark dependency jar but it cannot find it in the > > class path upon being submitted to spark. > > > > In the instructions I couldn't find how to explicitly add the > mahout-spark > > dependency jar to the class path. > > > > The question is: Am I doing the configurations correctly or not? > > > > Sorry for the lengthy email > > > > Kind Regards, > > Bahaa > > >
