Re: running mahout on Hadoop 2.0.0-cdh4.3.1
Hi Rohit I also run into the IncompatibleClassChangeError when using the streaming k means feature. Apparently this only applies to some parts of the code base because I have not run into this issue with other distributed functionalities of mahout. Were you able to solve this? If so, what were the specific parameters that you used to build mahout? I'm using CDH4.3.0-mr1 client and cluster. Kind regards, Barrie 2013/9/12 Gokhan Capan gkhn...@gmail.com You might also need to build mahout using hadoop-0.23 profile with hadoop.version parameter set to your hadoop version. Gokhan On Tue, Sep 10, 2013 at 10:09 PM, Parimi Rohit rohit.par...@gmail.com wrote: Thanks Sean. Will look into that. Rohit On Tue, Sep 10, 2013 at 1:32 PM, Sean Owen sro...@gmail.com wrote: You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1 and related branches. This wont work. However the CDH distributions also come in an 'mr1' flavor that stands a much better chance of working with something that is built for Hadoop 1. Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now) You will likely still have to compile Mahout again with this different dependency to get it to work but with any luck that's it. Sean On Sep 10, 2013 6:34 PM, Parimi Rohit rohit.par...@gmail.com wrote: Hi All, I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the Apache Hadoop environment, however, we had to switch to Cloudera distribution. When I try to run the item based collaborative filtering job (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the Cloudera distribution, I get the following error, Exception in thread main java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174) at org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614) at org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Is there a way to run Mahout in the Cloudera environment? I mean, a download specific to Cloudera distribution of Hadoop? Thanks in advance, Rohit
Re: running mahout on Hadoop 2.0.0-cdh4.3.1
Hi Barrie, We were not able to solve this issue. Hence we decided to install Apache hadoop 1.04 version. Thanks, Rohit On Wed, Sep 18, 2013 at 8:32 AM, B Kersbergen kersberg...@gmail.com wrote: Hi Rohit I also run into the IncompatibleClassChangeError when using the streaming k means feature. Apparently this only applies to some parts of the code base because I have not run into this issue with other distributed functionalities of mahout. Were you able to solve this? If so, what were the specific parameters that you used to build mahout? I'm using CDH4.3.0-mr1 client and cluster. Kind regards, Barrie 2013/9/12 Gokhan Capan gkhn...@gmail.com You might also need to build mahout using hadoop-0.23 profile with hadoop.version parameter set to your hadoop version. Gokhan On Tue, Sep 10, 2013 at 10:09 PM, Parimi Rohit rohit.par...@gmail.com wrote: Thanks Sean. Will look into that. Rohit On Tue, Sep 10, 2013 at 1:32 PM, Sean Owen sro...@gmail.com wrote: You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1 and related branches. This wont work. However the CDH distributions also come in an 'mr1' flavor that stands a much better chance of working with something that is built for Hadoop 1. Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now) You will likely still have to compile Mahout again with this different dependency to get it to work but with any luck that's it. Sean On Sep 10, 2013 6:34 PM, Parimi Rohit rohit.par...@gmail.com wrote: Hi All, I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the Apache Hadoop environment, however, we had to switch to Cloudera distribution. When I try to run the item based collaborative filtering job (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the Cloudera distribution, I get the following error, Exception in thread main java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174) at org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614) at org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Is there a way to run Mahout in the Cloudera environment? I mean, a download specific to Cloudera distribution of Hadoop? Thanks in advance, Rohit
Re: running mahout on Hadoop 2.0.0-cdh4.3.1
You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1 and related branches. This wont work. However the CDH distributions also come in an 'mr1' flavor that stands a much better chance of working with something that is built for Hadoop 1. Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now) You will likely still have to compile Mahout again with this different dependency to get it to work but with any luck that's it. Sean On Sep 10, 2013 6:34 PM, Parimi Rohit rohit.par...@gmail.com wrote: Hi All, I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the Apache Hadoop environment, however, we had to switch to Cloudera distribution. When I try to run the item based collaborative filtering job (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the Cloudera distribution, I get the following error, Exception in thread main java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174) at org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614) at org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Is there a way to run Mahout in the Cloudera environment? I mean, a download specific to Cloudera distribution of Hadoop? Thanks in advance, Rohit
Re: running mahout on Hadoop 2.0.0-cdh4.3.1
Thanks Sean. Will look into that. Rohit On Tue, Sep 10, 2013 at 1:32 PM, Sean Owen sro...@gmail.com wrote: You are trying to run on Hadoop 2 and Mahout only works with Hadoop 1 and related branches. This wont work. However the CDH distributions also come in an 'mr1' flavor that stands a much better chance of working with something that is built for Hadoop 1. Use 2.0.0-mr1-4.3.1 instead. (PS 4.3.2 and 4.4.0 are available now) You will likely still have to compile Mahout again with this different dependency to get it to work but with any luck that's it. Sean On Sep 10, 2013 6:34 PM, Parimi Rohit rohit.par...@gmail.com wrote: Hi All, I am used to running mahout (mahout-core-0.9-SNAPSHOT-job.jar) in the Apache Hadoop environment, however, we had to switch to Cloudera distribution. When I try to run the item based collaborative filtering job (org.apache.mahout.cf.taste.hadoop.item.RecommenderJob) in the Cloudera distribution, I get the following error, Exception in thread main java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.mahout.common.HadoopUtil.getCustomJobName(HadoopUtil.java:174) at org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614) at org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob.java:75) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:158) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:312) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Is there a way to run Mahout in the Cloudera environment? I mean, a download specific to Cloudera distribution of Hadoop? Thanks in advance, Rohit
