Hey- sorry for long delay. I've been traveling. Pat Ferrel was telling me he was having some simlar issues with Spark+Mahout+SBT recently, and that we need to re-examine our naming conventions on JARs.
Fwiw- I have several project that use Spark+Mahout in Spark 2.1/Scala-2.11, and we even test this in our Travis CI tests, but the trick is- we use Maven for the build. Any chance you could use maven? If not, maybe Pat can chime in here, I'm just not an SBT user, so I'm not 100% sure what to tell you. On Fri, Sep 22, 2017 at 10:06 PM, Hoa Nguyen <h...@insightdatascience.com> wrote: > Hey all, > > Thanks for the offers of help. I've been able to narrow down some of the > problems to version incompatibility and I just wanted to give an update. > Just to back track a bit, my initial goal was to run Mahout on a > distributed cluster whether that was running Hadoop Map Reduce or Spark. > > I started out trying to get it to run on Spark, which I have some > familiarity, but that didn't seem to work. While the error messages seem to > indicate there weren't enough resources on the workers ("WARN > scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; > check your cluster UI to ensure that workers are registered and have > sufficient memory"), I'm pretty sure that wasn't the case, not only because > it's a 4 node cluster of m4.xlarges, I was able to run another, simpler > Spark batch job on that same distributed cluster. > > After a bit of wrangling, I was able to narrow down some of the issues. It > turns out I was kind of blindly using this repo https://github.com/ > pferrel/3-input-cooc as a guide without fully realizing that it was from > several years ago and based on Mahout 0.10.0, Scala 2.10 and Spark 1.1.1 > That is significantly different from my environment, which has Mahout > 0.13.0 and Spark 2.1.1 installed, which also means I have to use Scala > 2.11. After modifying the build.sbt file to account for those versions, I > now have compile type mismatch issues that I'm just not that savvy to fix > (see attached screenshot if you're interested). > > Anyway, the good news that I was able to finally get Mahout code running > on Hadoop map-reduce, but also after a bit wrangling. It turned out my > instances were running Ubuntu 14 and apparently that doesn't play well with > Hadoop 2.7.4, which prevented me from running any sample Mahout code (from > here: https://github.com/apache/mahout/tree/master/examples/bin) that > relied on map-reduce. Those problems went away after I installed Hadoop > 2.8.1 instead. Now I'm able to get the shell scripts running on a > distributed Hadoop cluster (yay!). > > Anyway, if anyone has more recent and working Spark Scala code that uses > Mahout that they can point me to, I'd appreciate it. > > Many thanks! > Hoa > > On Fri, Sep 22, 2017 at 1:09 AM, Trevor Grant <trevor.d.gr...@gmail.com> > wrote: > >> Hi Hoa, >> >> A few things could be happening here, I haven't run across that specific >> error. >> >> 1) Spark 2.x - Mahout 0.13.0: Mahout 0.13.0 WILL run on Spark 2.x, however >> you need to build from source (not the binaries). You can do this by >> downloading mahout source or cloning the repo and building with: >> mvn clean install -Pspark-2.1,scala-2.11 -DskipTests >> >> 2) Have you setup spark with Kryo serialization? How you do this depends >> on >> if you're in the shell/zeppelin or using spark submit. >> >> However, for both of these cases- it shouldn't have even run local afaik >> so >> the fact it did tells me you probably have gotten this far? >> >> Assuming you've done 1 and 2, can you share some code? I'll see if I can >> recreate on my end. >> >> Thanks! >> >> tg >> >> On Thu, Sep 21, 2017 at 9:37 PM, Hoa Nguyen <h...@insightdatascience.com> >> wrote: >> >> > I apologize in advance if this is too much of a newbie question but I'm >> > having a hard time running any Mahout example code in a distributed >> Spark >> > cluster. The code runs as advertised when Spark is running locally on >> one >> > machine but the minute I point Spark to a cluster and master url, I >> can't >> > get it to work, drawing the error: "WARN scheduler.TaskSchedulerImpl: >> > Initial job has not accepted any resources; check your cluster UI to >> ensure >> > that workers are registered and have sufficient memory" >> > >> > I know my Spark cluster is configured and working correctly because I >> ran >> > non-Mahout code and it runs on a distributed cluster fine. What am I >> doing >> > wrong? The only thing I can think of is that my Spark version is too >> recent >> > -- 2.1.1 -- for the Mahout version I'm using -- 0.13.0. Is that it or >> am I >> > doing something else wrong? >> > >> > Thanks for any advice, >> > Hoa >> > >> > >