Would the HBase bug or error CRUNCH-141 be also because of the Hadoop version being pre-1.0.x?
________________________________ From: Josh Wills <[email protected]> To: Deepak Vohra <[email protected]> Cc: "[email protected]" <[email protected]> Sent: Thursday, January 10, 2013 5:10:04 PM Subject: Re: CRUNCH-140 Hrm-- I think that's b/c you're using an earlier version of Hadoop (pre-1.0.x) that doesn't support multiple outputs, which Crunch relies on. On Thu, Jan 10, 2013 at 5:08 PM, Deepak Vohra <[email protected]> wrote: From userlogs: > > >WARN org.apache.hadoop.mapred.Child: Error running child >org.apache.crunch.impl.mr.run.CrunchRuntimeException: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to create file /tmp/crunch-531439899/p1/output/_temporary/_attempt_201301102312_0002_r_000000_3/part-r-00000 for DFSClient_attempt_201301102312_0002_r_000000_3 on client 10.210.42.32 because current leaseholder is trying to recreate file. > > > > > > > > > >________________________________ > From: Josh Wills <[email protected]> >To: Deepak Vohra <[email protected]> >Sent: Thursday, January 10, 2013 4:36:36 PM >Subject: Re: CRUNCH-140 > > > >K-- any info in the logs for the job? > > > >On Thu, Jan 10, 2013 at 4:35 PM, Deepak Vohra <[email protected]> wrote: > >Yes. The input is required to be on the HDFS. >> >> >>But, the Crunch job still fails with the following output. >>13/01/11 00:10:45 INFO collect.PGroupedTableImpl: Setting num reduce tasks to 1 >>13/01/11 00:10:48 INFO input.FileInputFormat: Total input paths to process : 1 >>13/01/11 00:10:48 INFO exec.CrunchJob: Running job "org.apache.crunch.examples.WordCount: Text(LICENSE.txt)+S0+Aggregate.count+GBK+combine+asText+Text(crunch)" >>13/01/11 00:10:48 INFO exec.CrunchJob: Job status available at: http://ec2-50-19-55-40.compute-1.amazonaws.com:50030/jobdetails.jsp?jobid=job_201301102312_0002 >>1 job failure(s) occurred: >>org.apache.crunch.examples.WordCount: Text(LICENSE.txt)+S0+Aggregate.count+GBK+combine+asText+Text(crunch)(class org.apache.crunch.examples.WordCount0): Job failed! >> >> >> >>________________________________ >> From: Josh Wills <[email protected]> >>To: [email protected]; Deepak Vohra <[email protected]> >>Sent: Thursday, January 10, 2013 1:26:35 PM >>Subject: Re: CRUNCH-140 >> >> >> >>Does running: >> >> >>hadoop fs -put LICENSE.txt . >> >> >>followed by hadoop jar ... fix it? >> >> >> >>On Thu, Jan 10, 2013 at 12:07 PM, Deepak Vohra <[email protected]> wrote: >> >>The command/s to test bug >>>https://issues.apache.org/jira/browse/CRUNCH-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13549900#comment-13549900 >>> >>>>cd /usr/local/apache-crunch-0.4.0-incubating-bin >>>>cp LICENSE LICENSE.txt >>>>sudo mkdir crunch >>> >>>>hadoop jar crunch-examples-0.4.0-incubating-job.jar >>>>org.apache.crunch.examples.WordCount >>>LICENSE.txt crunch >> >> >> > > >
