You may not need to, if u build Mahout with Hadoop 2 profile u should be good. Please look at Gokhan's email and try applying his patch.
On Thursday, February 20, 2014 5:25 PM, "Zhang, Pengchu" <pzh...@sandia.gov> wrote: Please provide the link for open a JIRA Hadoop. -----Original Message----- From: Suneel Marthi [mailto:suneel_mar...@yahoo.com] Sent: Thursday, February 20, 2014 2:35 PM To: user@mahout.apache.org Subject: Re: [EXTERNAL] Re: Mapreduce job failed On Thursday, February 20, 2014 4:26 PM, "Zhang, Pengchu" <pzh...@sandia.gov> wrote: Thanks, it has been executed successfully. Two more questions related to this: 1. This means that I have to execute Mahout for further analysis with the non-MR mode? No, that's not the case. There have been API changes between Hadoop 2.x and Hadoop 1.x. Mahout is not certified for Hadoop 2.x. But most of Mahout's jobs work on Hadoop 2.x, seqdirectory could be one job that's failing due to API incompatibility between Hadoop 1.x and Hadoop 2.x 2. It is too bad that Hadoop2.2. does not support for newer versions of Mahout. Are you aware of that Hadoop 1.x working with Mahout 0.8 0r 0.9 on MR? I do have a large dataset to be clustered. As mentioned earlier, Mahout 0.8/0.9 are certified for Hadoop 1.x so u shouldn't be seeing any issues with that. Could u open a JIRA for this issue so that its trackable? As we r now working towards Mahout 1.0 and Hadoop 2.x compatibility its good that u have reported this issue. Thanks. Thanks. Pengchu -----Original Message----- From: Suneel Marthi [mailto:suneel_mar...@yahoo.com] Sent: Thursday, February 20, 2014 1:17 PM To: user@mahout.apache.org Subject: [EXTERNAL] Re: Mapreduce job failed ... and the reason for this failing is that 'TaskAttemptContext' which was a Class in Hadoop 1.x has now become an interface in Hadoop 2.2. Suggest that u execute this job in non-MR mode with '-xm sequential'. On Thursday, February 20, 2014 2:26 PM, Suneel Marthi <suneel_mar...@yahoo.com> wrote: Seems like u r running this on HAdoop 2.2 (officially not supported for Mahout 0.8 or 0.9), work around is to run this in sequential mode with "-xm sequential". On Thursday, February 20, 2014 1:36 PM, "Zhang, Pengchu" <pzh...@sandia.gov> wrote: Hello, I am trying to "seqdirirectory" with mahout (0.8 and 0.9) on my Linux box with Hadoop (2.2.0) but keeping failed consistently. I tested Hadoop with the Hadoop example pi and wordcount, both worked well. With a simple text file or directory with multiple text files, e.g., Shakespeare_text, I got the same message of failure. >mahout seqdirectory --input /Shakespeare_txet --output >/Shakespeare-seqdir --charset utf-8 $ mahout seqdirectory --input /shakespeare_text --output /shakespeare-seqdir --charset utf-8 MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath. Running on hadoop, using /home/pzhang/hadoop-2.2.0/bin/hadoop and HADOOP_CONF_DIR=/home/pzhang/hadoop-2.2.0/etc/hadoop MAHOUT-JOB: /home/pzhang/MAHOUT_HOME/mahout-examples-0.8-job.jar 14/02/20 11:29:42 INFO common.AbstractJob: Command line arguments: {--charset=[utf-8], --chunkSize=[64], --endPhase=[2147483647], --fileFilterClass=[org.apache.mahout.text.PrefixAdditionFilter], --input=[/shakespeare_text], --keyPrefix=[], --method=[mapreduce], --output=[/shakespeare-seqdir], --startPhase=[0], --tempDir=[temp]} 14/02/20 11:29:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 14/02/20 11:29:42 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 14/02/20 11:29:42 INFO Configuration.deprecation: mapred.compress.map.output is deprecated. Instead, use mapreduce.map.output.compress 14/02/20 11:29:42 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 14/02/20 11:29:43 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 14/02/20 11:29:44 INFO input.FileInputFormat: Total input paths to process : 43 14/02/20 11:29:44 INFO input.CombineFileInputFormat: DEBUG: Terminated node allocation with : CompletedNodes: 1, size left: 5284832 14/02/20 11:29:44 INFO mapreduce.JobSubmitter: number of splits:1 14/02/20 11:29:44 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class 14/02/20 11:29:44 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name 14/02/20 11:29:44 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 14/02/20 11:29:44 INFO Configuration.deprecation: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class 14/02/20 11:29:44 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 14/02/20 11:29:44 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1392919123773_0004 14/02/20 11:29:45 INFO impl.YarnClientImpl: Submitted application application_1392919123773_0004 to ResourceManager at /0.0.0.0:8032 14/02/20 11:29:45 INFO mapreduce.Job: The url to track the job: http://savm0072lx.sandia.gov:8088/proxy/application_1392919123773_0004/ 14/02/20 11:29:45 INFO mapreduce.Job: Running job: job_1392919123773_0004 14/02/20 11:29:53 INFO mapreduce.Job: Job job_1392919123773_0004 running in uber mode : false 14/02/20 11:29:53 INFO mapreduce.Job: map 0% reduce 0% 14/02/20 11:29:58 INFO mapreduce.Job: Task Id : attempt_1392919123773_0004_m_000000_0, Status : FAILED Error: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.initNextRecordReader(CombineFileRecordReader.java:164) at org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.<init>(CombineFileRecordReader.java:126) at org.apache.mahout.text.MultipleTextFileInputFormat.createRecordReader(MultipleTextFileInputFormat.java:43) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:491) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:734) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:534) at org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.initNextRecordReader(CombineFileRecordReader.java:155) ... 10 more Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected at org.apache.mahout.text.WholeFileRecordReader.<init>(WholeFileRecordReader.java:52) ... 15 more Any suggestion is helpful. Thanks. Pengchu