Hi Sean, I am trying to build the Mahout again and got some WARNINGs so far. Can you give me some hints what I have done wrong here?
Thanks for your help so far. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ $ mv mahout-distribution-0.9 mahout-distribution-0.9.old $ tar xvf mahout-distribution-0.9-src.tar.gz $ cd mahout-distribution-0.9 $ ls bin buildtools core distribution examples integration LICENSE.txt math math-scala NOTICE.txt pom.xml README.txt src $ mvn clean install -Dhadoop2.version=2.2.0-cdh5.0.0-beta-1 -DskipTests=true [INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.mahout:mahout-math:jar:0.9 [WARNING] 'parent.relativePath' of POM org.apache.mahout:mahout:0.9 (/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml) points at org.apache.mahout:mahout-distribution instead of org.apache:apache, please verify your project structure @ org.apache.mahout:mahout:0.9, /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml, line 25, column 11 [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.mahout:mahout-core:jar:0.9 [WARNING] 'parent.relativePath' of POM org.apache.mahout:mahout:0.9 (/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml) points at org.apache.mahout:mahout-distribution instead of org.apache:apache, please verify your project structure @ org.apache.mahout:mahout:0.9, /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml, line 25, column 11 [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.mahout:mahout-integration:jar:0.9 [WARNING] 'parent.relativePath' of POM org.apache.mahout:mahout:0.9 (/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml) points at org.apache.mahout:mahout-distribution instead of org.apache:apache, please verify your project structure @ org.apache.mahout:mahout:0.9, /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml, line 25, column 11 [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.mahout:mahout-examples:jar:0.9 [WARNING] 'parent.relativePath' of POM org.apache.mahout:mahout:0.9 (/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml) points at org.apache.mahout:mahout-distribution instead of org.apache:apache, please verify your project structure @ org.apache.mahout:mahout:0.9, /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml, line 25, column 11 [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.mahout:mahout-distribution:pom:0.9 [WARNING] 'parent.relativePath' of POM org.apache.mahout:mahout:0.9 (/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml) points at org.apache.mahout:mahout-distribution instead of org.apache:apache, please verify your project structure @ org.apache.mahout:mahout:0.9, /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml, line 25, column 11 [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.mahout:mahout-math-scala:jar:0.9 [WARNING] 'parent.relativePath' of POM org.apache.mahout:mahout:0.9 (/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml) points at org.apache.mahout:mahout-distribution instead of org.apache:apache, please verify your project structure @ org.apache.mahout:mahout:0.9, /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/pom.xml, line 25, column 11 [WARNING] 'build.plugins.plugin.version' for org.scala-tools:maven-scala-plugin is missing. @ line 108, column 15 [WARNING] [WARNING] Some problems were encountered while building the effective model for org.apache.mahout:mahout:pom:0.9 [WARNING] 'parent.relativePath' points at org.apache.mahout:mahout-distribution instead of org.apache:apache, please verify your project structure @ line 25, column 11 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] Thanks and Regards, Truong Phan P + 61 2 8576 5771 M + 61 4 1463 7424 E troung.p...@team.telstra.com W www.telstra.com -----Original Message----- From: Sean Owen [mailto:sro...@gmail.com] Sent: Wednesday, 2 April 2014 9:53 AM To: Phan, Truong Q Cc: Mahout User List Subject: Re: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1 This still means Hadoop version mismatch. You shouldn't have to use CLASSPATH. To be safe use the .job file with all the dependencies including Hadoop client baked in. I am guessing that you are picking up other versions of Mahout which are not compiled for Hadoop 2, in that CLASSPATH? Kind of a guess though. Your build invocation should set the Hadoop 2 profile and all that. On Wed, Apr 2, 2014 at 12:41 AM, Phan, Truong Q <troung.p...@team.telstra.com> wrote: > Hi Sean, > > Thanks for reviewing my issue. > Actually, the directory (/usr/lib/hadoop-0.20-mapreduce) contain Cloudera > v5b1 which I believe it has been upgraded to Hadoop 2.2. > I also have another Hadoop home directory (/usr/lib/hadoop-mapreduce/) which > is also belong to Cloudera v5b1. > Please see below for the content of these directories. > > For the sake of this testing I have pointed my CLASSPATH to > (/usr/lib/hadoop-mapreduce/) and re-run the test again but I receive a > different error. > Please see below for the error. > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > $ env | grep HAD > HADOOP_CMD=/usr/bin/hadoop > HADOOP_HOME=/usr/lib/hadoop > HADOOP_STREAMING=/usr/lib/hadoop-mapreduce/hadoop-streaming-2.2.0-cdh5 > .0.0-beta-1.jar > > $ echo $CLASSPATH > :/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distri > bution-0.9/core/target/mahout-core-0.9.jar:/ora/db002/stg001/BDMSL1D/h > adoop/nem-dms/devices/mahout/mahout-distribution-0.9/core/target/mahou > t-core-0.9-job.jar:/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/ma > hout/mahout-distribution-0.9/core/target/mahout-core-0.9-sources.jar:/ > ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribu > tion-0.9/core/target/mahout-core-0.9-tests.jar:/ora/db002/stg001/BDMSL > 1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/math/target/m > ahout-math-0.9.jar:/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/ma > hout/mahout-distribution-0.9/math/target/mahout-math-0.9-sources.jar:/ > ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribu > tion-0.9/math/target/mahout-math-0.9-tests.jar:/ora/db002/stg001/BDMSL > 1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/integration/t > arget/mahout-integration-0.9.jar:/ora/db002/stg001/BDMSL1D/hadoop/nem- > dms/devices/mahout/mahout-distribution-0.9/integration/target/mahout-i > ntegration-0.9-sources.jar > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > $ mahout recommenditembased --input mydata.dat --usersFile user.dat > --numRecommendations 2 --output output/ --similarityClassname > SIMILARITY_PEARSON_CORRELATION Running on hadoop, using > /usr/lib/hadoop/bin/hadoop and HADOOP_CONF_DIR= > MAHOUT-JOB: > /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distrib > ution-0.9/examples/target/mahout-examples-0.9-job.jar > 14/04/01 17:17:06 INFO common.AbstractJob: Command line arguments: > {--booleanData=[false], --endPhase=[2147483647], --input=[mydata.dat], > --maxPrefsInItemSimilarity=[500], --maxPrefsPerUser=[10], > --maxSimilaritiesPerItem=[100], --minPrefsPerUser=[1], > --numRecommendations=[2], --output=[output/], > --similarityClassname=[SIMILARITY_PEARSON_CORRELATION], > --startPhase=[0], --tempDir=[temp], --usersFile=[user.dat]} > 14/04/01 17:17:06 INFO common.AbstractJob: Command line arguments: > {--booleanData=[false], --endPhase=[2147483647], --input=[mydata.dat], > --minPrefsPerUser=[1], --output=[temp/preparePreferenceMatrix], > --ratingShift=[0.0], --startPhase=[0], --tempDir=[temp]} > 14/04/01 17:17:07 INFO Configuration.deprecation: mapred.input.dir is > deprecated. Instead, use mapreduce.input.fileinputformat.inputdir > 14/04/01 17:17:07 INFO Configuration.deprecation: > mapred.compress.map.output is deprecated. Instead, use > mapreduce.map.output.compress > 14/04/01 17:17:07 INFO Configuration.deprecation: mapred.output.dir is > deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir > Exception in thread "main" java.lang.IncompatibleClassChangeError: Found > interface org.apache.hadoop.mapreduce.JobContext, but class was expected > at > org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174) > at > org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614) > at > org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:73) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:164) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:322) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) > at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) > at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152) > at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > $ ls -al /usr/lib/hadoop-0.20-mapreduce total 3860 > drwxr-xr-x 10 root root 4096 Feb 5 07:08 . > drwxr-xr-x 52 root root 12288 Feb 10 16:56 .. > drwxr-xr-x 2 root root 4096 Feb 12 04:02 bin > -rw-r--r-- 1 root root 348776 Oct 28 11:30 CHANGES.txt > drwxr-xr-x 2 root root 4096 Feb 5 07:08 cloudera > lrwxrwxrwx 1 root root 16 Feb 5 07:08 conf -> /etc/hadoop/conf > drwxr-xr-x 6 root root 4096 Feb 5 07:08 contrib > drwxr-xr-x 4 root root 4096 Feb 5 07:08 example-confs > -rw-r--r-- 1 root root 6833 Oct 28 11:30 > hadoop-ant-2.2.0-mr1-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 40 Feb 5 07:08 hadoop-ant.jar -> > hadoop-ant-2.2.0-mr1-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 1757210 Oct 28 11:30 > hadoop-core-2.2.0-mr1-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 41 Feb 5 07:08 hadoop-core.jar -> > hadoop-core-2.2.0-mr1-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 142842 Oct 28 11:30 > hadoop-examples-2.2.0-mr1-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 45 Feb 5 07:08 hadoop-examples.jar -> > hadoop-examples-2.2.0-mr1-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 1413120 Oct 28 11:30 > hadoop-test-2.2.0-mr1-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 41 Feb 5 07:08 hadoop-test.jar -> > hadoop-test-2.2.0-mr1-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 175182 Oct 28 11:30 > hadoop-tools-2.2.0-mr1-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 42 Feb 5 07:08 hadoop-tools.jar -> > hadoop-tools-2.2.0-mr1-cdh5.0.0-beta-1.jar > drwxr-xr-x 3 root root 4096 Feb 5 07:08 include > drwxr-xr-x 5 root root 4096 Feb 5 07:08 lib > -rw-r--r-- 1 root root 13366 Oct 28 11:30 LICENSE.txt > -rw-r--r-- 1 root root 101 Oct 28 11:30 NOTICE.txt > -rw-r--r-- 1 root root 1366 Oct 28 11:30 README.txt > drwxr-xr-x 3 root root 4096 Feb 5 07:08 sbin > drwxr-xr-x 5 root root 4096 Feb 5 07:08 webapps > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > $ ls -al /usr/lib/hadoop-mapreduce/ > total 5244 > drwxr-xr-x 6 root root 4096 Feb 5 07:05 . > drwxr-xr-x 52 root root 12288 Feb 10 16:56 .. > drwxr-xr-x 2 root root 4096 Feb 12 04:02 bin > drwxr-xr-x 2 root root 4096 Feb 5 07:05 cloudera > -rw-r--r-- 1 root root 21515 Oct 28 11:30 > hadoop-archives-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 41 Feb 5 07:05 hadoop-archives.jar -> > hadoop-archives-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 14574 Oct 28 11:30 > hadoop-datajoin-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 41 Feb 5 07:05 hadoop-datajoin.jar -> > hadoop-datajoin-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 80406 Oct 28 11:30 > hadoop-distcp-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 39 Feb 5 07:05 hadoop-distcp.jar -> > hadoop-distcp-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 62071 Oct 28 11:30 > hadoop-extras-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 39 Feb 5 07:05 hadoop-extras.jar -> > hadoop-extras-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 215375 Oct 28 11:30 > hadoop-gridmix-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 40 Feb 5 07:05 hadoop-gridmix.jar -> > hadoop-gridmix-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 482199 Oct 28 11:30 > hadoop-mapreduce-client-app-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 53 Feb 5 07:05 hadoop-mapreduce-client-app.jar > -> hadoop-mapreduce-client-app-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 656300 Oct 28 11:30 > hadoop-mapreduce-client-common-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 56 Feb 5 07:05 > hadoop-mapreduce-client-common.jar -> > hadoop-mapreduce-client-common-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 1455612 Oct 28 11:30 > hadoop-mapreduce-client-core-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 54 Feb 5 07:05 hadoop-mapreduce-client-core.jar > -> hadoop-mapreduce-client-core-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 117249 Oct 28 11:30 > hadoop-mapreduce-client-hs-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 52 Feb 5 07:05 hadoop-mapreduce-client-hs.jar > -> hadoop-mapreduce-client-hs-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 4086 Oct 28 11:30 > hadoop-mapreduce-client-hs-plugins-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 60 Feb 5 07:05 > hadoop-mapreduce-client-hs-plugins.jar -> > hadoop-mapreduce-client-hs-plugins-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 35237 Oct 28 11:30 > hadoop-mapreduce-client-jobclient-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 1434812 Oct 28 11:30 > hadoop-mapreduce-client-jobclient-2.2.0-cdh5.0.0-beta-1-tests.jar > lrwxrwxrwx 1 root root 59 Feb 5 07:05 > hadoop-mapreduce-client-jobclient.jar -> > hadoop-mapreduce-client-jobclient-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 21566 Oct 28 11:30 > hadoop-mapreduce-client-shuffle-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 57 Feb 5 07:05 > hadoop-mapreduce-client-shuffle.jar -> > hadoop-mapreduce-client-shuffle-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 270285 Oct 28 11:30 > hadoop-mapreduce-examples-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 51 Feb 5 07:05 hadoop-mapreduce-examples.jar -> > hadoop-mapreduce-examples-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 277597 Oct 28 11:30 > hadoop-rumen-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 38 Feb 5 07:05 hadoop-rumen.jar -> > hadoop-rumen-2.2.0-cdh5.0.0-beta-1.jar > -rw-r--r-- 1 root root 102813 Oct 28 11:30 > hadoop-streaming-2.2.0-cdh5.0.0-beta-1.jar > lrwxrwxrwx 1 root root 42 Feb 5 07:05 hadoop-streaming.jar -> > hadoop-streaming-2.2.0-cdh5.0.0-beta-1.jar > drwxr-xr-x 2 root root 4096 Feb 5 07:05 lib > drwxr-xr-x 2 root root 4096 Feb 5 07:05 sbin > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > Thanks and Regards, > Truong Phan > > > P + 61 2 8576 5771 > M + 61 4 1463 7424 > E troung.p...@team.telstra.com > W www.telstra.com > > > > -----Original Message----- > From: Sean Owen [mailto:sro...@gmail.com] > Sent: Monday, 31 March 2014 7:05 PM > To: Mahout User List > Subject: RE: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1 > > But you have a bunch of Hadoop 0.20 jars on your classpath! Definitely a > problem. Those should not be there. > On Mar 31, 2014 7:09 AM, "Phan, Truong Q" > <troung.p...@team.telstra.com> > wrote: > >> Yes, I did rebuild it. >> >> oracle@bpdevdmsdbs01: ---- >> /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distri >> b >> ution-0.9 >> ----- >> $ mvn clean install -Dhadoop2.version=2.2.0-cdh5.0.0-beta-1 >> -DskipTests=true >> [INFO] Scanning for projects... >> .... >> [INFO] >> --------------------------------------------------------------------- >> - >> -- >> [INFO] Reactor Summary: >> [INFO] >> [INFO] Mahout Build Tools ................................ SUCCESS [ >> 8.215 s] >> [INFO] Apache Mahout ..................................... SUCCESS [ >> 1.158 s] >> [INFO] Mahout Math ....................................... SUCCESS >> [16:21 min] [INFO] Mahout Core ....................................... >> SUCCESS [26:21 min] [INFO] Mahout Integration >> ................................ SUCCESS [03:55 min] [INFO] Mahout >> Examples ................................... SUCCESS [02:54 min] >> [INFO] Mahout Release Package ............................ SUCCESS [ >> 0.084 s] >> [INFO] Mahout Math/Scala wrappers ........................ SUCCESS >> [01:16 min] [INFO] >> --------------------------------------------------------------------- >> - >> -- >> [INFO] BUILD SUCCESS >> [INFO] >> --------------------------------------------------------------------- >> - >> -- >> [INFO] Total time: 50:59 min >> [INFO] Finished at: 2014-03-31T14:25:27+10:00 [INFO] Final Memory: >> 47M/250M [INFO] >> --------------------------------------------------------------------- >> - >> -- >> >> >> Thanks and Regards, >> Truong Phan >> >> >> P + 61 2 8576 5771 >> M + 61 4 1463 7424 >> E troung.p...@team.telstra.com >> W www.telstra.com >> >> >> -----Original Message----- >> From: Andrew Musselman [mailto:andrew.mussel...@gmail.com] >> Sent: Monday, 31 March 2014 2:44 PM >> To: user@mahout.apache.org >> Subject: Re: Mahout v0.9 is not working with 2.2.0-cdh5.0.0-beta-1 >> >> Have you rebuilt Mahout for your version? We're not supporting >> Hadoop version two yet. >> >> See here for some direction: >> http://mail-archives.us.apache.org/mod_mbox/mahout-user/201403.mbox/% >> 3 >> CCANg8BGD8Cm_=ESecQQ5mDL+6ybbNrR1Ce7i=pkuimxmcktw...@mail.gmail.com%3 >> E >> >> > On Mar 30, 2014, at 7:28 PM, "Phan, Truong Q" < >> troung.p...@team.telstra.com> wrote: >> > >> > Hi >> > >> > Does Mahout v0.9 supports Cloudera Hadoop v5 (2.2.0-cdh5.0.0-beta-1)? >> > I have managed to installed and run all test cases under the Mahout >> > v0.9 >> without any issue. >> > Please see below for the evident of the test cases. >> > However I have no success to run the example from >> http://girlincomputerscience.blogspot.com.au/2010/11/apache-mahout.htmland >> got the following errors. >> > Note: I have set the CLASSPATH to point to all of Mahout’s jar files. >> > >> > <snip> >> > $ env | grep CLASS >> > CLASSPATH=:/usr/lib/hadoop-0.20-mapreduce/lib:/usr/lib/hadoop-0.20- >> > m >> > ap >> > reduce/lib:/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/ >> > m >> > ah >> > out-distribution-0.9/core/target/mahout-core-0.9.jar:/ora/db002/stg >> > 0 >> > 01 >> > /BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/core >> > / >> > ta >> > rget/mahout-core-0.9-job.jar:/ora/db002/stg001/BDMSL1D/hadoop/nem-d >> > m >> > s/ >> > devices/mahout/mahout-distribution-0.9/core/target/mahout-core-0.9- >> > s >> > ou >> > rces.jar:/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/ma >> > h >> > ou >> > t-distribution-0.9/core/target/mahout-core-0.9-tests.jar:/ora/db002 >> > / >> > st >> > g001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/ >> > m >> > at >> > h/target/mahout-math-0.9.jar:/ora/db002/stg001/BDMSL1D/hadoop/nem-d >> > m >> > s/ >> > devices/mahout/mahout-distribution-0.9/math/target/mahout-math-0.9- >> > s >> > ou >> > rces.jar:/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/ma >> > h >> > ou >> > t-distribution-0.9/math/target/mahout-math-0.9-tests.jar:/ora/db002 >> > / >> > st >> > g001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-distribution-0.9/ >> > i >> > nt >> > egration/target/mahout-integration-0.9.jar:/ora/db002/stg001/BDMSL1 >> > D >> > /h >> > adoop/nem-dms/devices/mahout/mahout-distribution-0.9/integration/ta >> > r ge t/mahout-integration-0.9-sources.jar >> > >> > $ export >> > MAHOUT_HOME=/ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout >> > / >> > ma >> > hout-distribution-0.9 >> > $ export PATH=$MAHOUT_HOME/bin:$PATH >> > >> > oracle@bpdevdmsdbs01:BDMSSI1D1 ---- >> > /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-dist >> > r ib ution-0.9/nem-dms ----- $ mahout recommenditembased --input >> > mydata.dat --usersFile user.dat --numRecommendations 2 --output >> > output/ --similarityClassname SIMILARITY_PEARSON_CORRELATION >> > Running on hadoop, using /usr/lib/hadoop-0.20-mapreduce/bin/hadoop >> > and HADOOP_CONF_DIR= >> > MAHOUT-JOB: >> > /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-dist >> > r ib ution-0.9/examples/target/mahout-examples-0.9-job.jar >> > Exception in thread "main" java.lang.NoClassDefFoundError: >> > org/apache/hadoop/util/PlatformName >> > Caused by: java.lang.ClassNotFoundException: >> org.apache.hadoop.util.PlatformName >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> > at java.security.AccessController.doPrivileged(Native Method) >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> > Could not find the main class: org.apache.hadoop.util.PlatformName. >> Program will exit. >> > Exception in thread "main" java.lang.NoClassDefFoundError: >> > org/apache/hadoop/util/RunJar Caused by: >> java.lang.ClassNotFoundException: org.apache.hadoop.util.RunJar >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> > at java.security.AccessController.doPrivileged(Native Method) >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> > Could not find the main class: org.apache.hadoop.util.RunJar. >> > Program >> will exit. >> > </snip> >> > >> > <snip> >> > [INFO] --- maven-install-plugin:2.5.1:install (default-install) @ >> > mahout-math-scala --- [INFO] Installing >> > /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-dist >> > r ib ution-0.9/math-scala/target/mahout-math-scala-0.9.jar to >> > /home/oracle/.m2/repository/org/apache/mahout/mahout-math-scala/0.9 >> > / ma hout-math-scala-0.9.jar [INFO] Installing >> > /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-dist >> > r >> > ib >> > ution-0.9/math-scala/pom.xml to >> > /home/oracle/.m2/repository/org/apache/mahout/mahout-math-scala/0.9 >> > / ma hout-math-scala-0.9.pom [INFO] Installing >> > /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-dist >> > r ib ution-0.9/math-scala/target/mahout-math-scala-0.9-tests.jar to >> > /home/oracle/.m2/repository/org/apache/mahout/mahout-math-scala/0.9 >> > / ma hout-math-scala-0.9-tests.jar [INFO] Installing >> > /ora/db002/stg001/BDMSL1D/hadoop/nem-dms/devices/mahout/mahout-dist >> > r ib ution-0.9/math-scala/target/mahout-math-scala-0.9-sources.jar >> > to >> > /home/oracle/.m2/repository/org/apache/mahout/mahout-math-scala/0.9 >> > / ma hout-math-scala-0.9-sources.jar [INFO] >> > ------------------------------------------------------------------- >> > - >> > -- >> > -- >> > [INFO] Reactor Summary: >> > [INFO] >> > [INFO] Mahout Build Tools ................................ SUCCESS >> > [01:21 min] [INFO] Apache Mahout ..................................... >> > SUCCESS [ 6.898 s] [INFO] Mahout Math >> > ....................................... SUCCESS [17:39 min] [INFO] >> > Mahout Core ....................................... SUCCESS [ >> > 01:01 h] [INFO] Mahout Integration ................................ >> > SUCCESS >> > [06:59 min] [INFO] Mahout Examples ................................... >> > SUCCESS [03:15 min] [INFO] Mahout Release Package >> > ............................ SUCCESS [ 0.026 s] [INFO] Mahout >> > Math/Scala wrappers ........................ SUCCESS [03:39 min] >> > [INFO] >> > ------------------------------------------------------------------- >> > - >> > -- >> > -- >> > [INFO] BUILD SUCCESS >> > [INFO] >> > ------------------------------------------------------------------- >> > - >> > -- >> > -- >> > [INFO] Total time: 01:34 h >> > [INFO] Finished at: 2014-03-28T13:10:48+10:00 [INFO] Final Memory: >> > 49M/276M [INFO] >> > ------------------------------------------------------------------- >> > - >> > -- >> > -- >> > </snip> >> > >> > Thanks and Regards, >> > Truong Phan >> > Senior Technology Specialist >> > Database Engineering >> > Transport & Routing Engineering | Networks | Telstra Operations >> > >> > P + 61 2 8576 5771 >> > M + 61 4 1463 7424 >> > E troung.p...@team.telstra.com >> > W www.telstra.com >> > Love the movies? Telstra takes you there with $10 movie tickets, >> > just to say thanks. Available now at telstra.com/movies This >> > communication >> may contain confidential or copyright information of Telstra >> Corporation Limited (ABN 33 051 775 556). If you are not an intended >> recipient, you must not keep, forward, copy, use, save or rely on >> this communication, and any such action is unauthorised and >> prohibited. If you have received this communication in error, please >> reply to this email to notify the sender of its incorrect delivery, and then >> delete both it and your reply. >> > >> > >>
