Did you build Spark from source and deploy it to the cluster? When you build Mahout it’s running its tests against the artifacts it gets from maven repos. When you run mahout on a cluster it is running from the artifacts on the cluster. These may not be the same and there have been problems that present similari to what you have shown. Building Spark from source has solved these for several people (including me).
I haven’t used yarn since I’m still on hadoop 1.2.1 but another user on this list was successful after some initial problems. Not sure if passing the yarn master in is sufficient there are a bunch of Spark-yarn config params you may want to check out and if any are runtime conf you can pass them in to the spark-itemsimilarity with the -D:key=value CLI option. On Jan 6, 2015, at 12:30 PM, Pasmanik, Paul <paul.pasma...@danteinc.com> wrote: So, when I follow examples from hortonworks and run spark Pi example using spark-submit - everything works. I can run mahout spark-itemsimilarity without specifying master parameter which means it is running in the local mode (right?) and it works. But if I try to run mahout using -ma (master) parameter to point to yarn cluster it always gets stuck with the following warning: WARN YarnClusterScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory According to several places the error means that Hadoop does not have sufficient memory - but I have plenty and I tried to lower executor-memory and driver-memory all way to 250 Megs. I still get that error and nothing is processed. Did you guys run into this issues? Thanks. More stack trace below: 15/01/06 12:14:57 INFO storage.MemoryStore: ensureFreeSpace(4024) called with curMem=87562, maxMem=2061647216 15/01/06 12:14:57 INFO storage.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.9 KB, free 1966.1 MB) 15/01/06 12:14:57 INFO storage.MemoryStore: ensureFreeSpace(2336) called with curMem=91586, maxMem=2061647216 15/01/06 12:14:57 INFO storage.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.3 KB, free 1966.1 MB) 15/01/06 12:14:57 INFO storage.BlockManagerInfo: Added broadcast_1_piece0 in memory on sandbox.hortonworks.com:53919 (size: 2.3 KB, free: 19 66.1 MB) 15/01/06 12:14:57 INFO storage.BlockManagerMaster: Updated info of block broadcast_1_piece0 15/01/06 12:14:57 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from Stage 1 (MappedRDD[6] at distinct at TextDelimitedReaderWrite r.scala:76) 15/01/06 12:14:57 INFO cluster.YarnClusterScheduler: Adding task set 1.0 with 2 tasks 15/01/06 12:14:57 INFO util.RackResolver: Resolved sandbox.hortonworks.com to /default-rack 15/01/06 12:15:13 WARN cluster.YarnClusterScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that worker s are registered and have sufficient memory 15/01/06 12:15:27 WARN cluster.YarnClusterScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that worker s are registered and have sufficient memory -----Original Message----- From: Pasmanik, Paul [mailto:paul.pasma...@danteinc.com] Sent: Tuesday, January 06, 2015 2:49 PM To: user@mahout.apache.org Subject: RE: running spark-itemsimilarity against HDP sandbox with Spark Thanks, Pat. I am using HDP with spark 1.1.0: http://hortonworks.com/hadoop-tutorial/using-apache-spark-hdp/ Spark examples run without issues. For mahout I had to create a couple of env vars: (HADOOP_HOME, SPARK_HOME, MAHOUT_HOME). Also, to run using yarn cluster with HDP -ma yarn-cluster needs to be passed in. Also, default memory allocated to yarn was not enough out of the box ( 2g), increased to 3g, now restarting and trying again. -----Original Message----- From: Pat Ferrel [mailto:p...@occamsmachete.com] Sent: Tuesday, January 06, 2015 12:58 PM To: user@mahout.apache.org Subject: Re: running spark-itemsimilarity against HDP sandbox with Spark There are some issues with using Mahout on Windows so you’ll have to run on a ‘nix machine or VM. There shouldn’t be any problem with using VMs as long as your Spark install is setup correctly. Currently you have to build Spark first and then Mahout from source. Mahout uses Spark 1.1. You’ll need to build Spark from source using “mvn install” rather than their recommended “mvn package” There were some problems in the Spark artifacts when running from the binary release. Check Mahout’s Spark FAQ for some pointers http://mahout.apache.org/users/sparkbindings/faq.html Verify Spark is running correctly by trying their sample SparkPi job. http://spark.apache.org/docs/1.1.1/submitting-applications.html Spark in general and spark-itemsimilarity especially like lots of memory so you may have to play with the -sem option to spark-itemsimilarity. On Jan 6, 2015, at 8:07 AM, Pasmanik, Paul <paul.pasma...@danteinc.com> wrote: Hi, I've been trying to run spark-itemsimilarity against Hortonworks Sandbox with Spark running in a VM, but have not succeeded yet. Do I need to install mahout and run within a VM or is there a way to run remotely against a VM where spark and hadoop are running? I tried running a scala ItemSimilaritySuite test with some modifications pointing hdfs and spark to sandbox but getting various errors the latest one with ShuffleMapTask getting hdfs block missing exception trying to read an input file that I uploaded to the hdfs cluster. ________________________________ The information contained in this electronic transmission is intended only for the use of the recipient and may be confidential and privileged. Unauthorized use, disclosure, or reproduction is strictly prohibited and may be unlawful. If you have received this electronic transmission in error, please notify the sender immediately.
