Thanks. That URL doesn't tell much. * * *Job Name:* Job387546913066708402.jar *Job File:* hdfs://hadoop-name01/hadoop/mapred/system/job_201101260357_3230/job.xml *Job Setup:*None *Status:* Failed *Started at:* Mon Jan 31 15:48:36 CST 2011 *Failed at:* Mon Jan 31 15:48:36 CST 2011 *Failed in:* 0sec *Job Cleanup:*None ------------------------------ Kind % CompleteNum Tasks PendingRunningComplete KilledFailed/Killed Task Attempts map 100.00% 0 0 0 0 0 0 / 0reduce 100.00% 0 0 0 0 0 0 / 0
CounterMap ReduceTotal ------------------------------ Map Completion Graph - close On Mon, Jan 31, 2011 at 2:37 PM, Thejas M Nair <te...@yahoo-inc.com> wrote: > The logs say that the map-reduce job failed. Can you check the log files > of the failed map-reduce tasks ? > You can follow the jobtracker url in the log message - * > http://hadoop-name02:50030/jobdetails.jsp?jobid=job_201101260357_3230 > * > -Thejas > > > > On 1/31/11 1:54 PM, "Dexin Wang" <wangde...@gmail.com> wrote: > > Hi, > > I found similar problems on the web but didn't find a solution for it so > I'm > asking here. > > I have some pig job that has been working fine for couple of months and it > started failing. But the same job still works if run as another account. I > narrowed it a bit and found that the problematic user account can't even do > a simple DUMP. > > > -------------------------------------------------------------------------------------- > grunt> A = LOAD '/user/myuser1/aa' AS (f1, f2); > grunt> DESCRIBE A; > A: {f1: bytearray,f2: bytearray} > grunt> DUMP A; > 2011-01-31 15:48:34,141 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: > > Store(hdfs://hadoop-name01/tmp/temp811847645/tmp1546738024:org.apache.pig.builtin.BinStorage) > - 1-10 Operator Key: 1-10) > 2011-01-31 15:48:34,142 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size before optimization: 1 > 2011-01-31 15:48:34,142 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size after optimization: 1 > 2011-01-31 15:48:34,153 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 > 2011-01-31 15:48:35,562 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - Setting up single store job > 2011-01-31 15:48:35,574 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map-reduce job(s) waiting for submission. > 2011-01-31 15:48:35,578 [Thread-23] WARN > org.apache.hadoop.mapred.JobClient > - Use GenericOptionsParser for parsing the arguments. Applications should > implement Tool for the same. > 2011-01-31 15:48:36,077 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 0% complete > 2011-01-31 15:48:36,176 [Thread-23] INFO > org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths > to process : 1 > 2011-01-31 15:48:36,176 [Thread-23] INFO > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total > input > paths to process : 1 > 2011-01-31 15:48:37,067 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - HadoopJobId: job_201101260357_3230 > 2011-01-31 15:48:37,067 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - More information at: > http://hadoop-name02:50030/jobdetails.jsp?jobid=job_201101260357_3230 > 2011-01-31 15:48:41,605 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 100% complete > 2011-01-31 15:48:41,605 [main] ERROR > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map reduce job(s) failed! > 2011-01-31 15:48:41,607 [main]* ERROR > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed to produce result in: > "hdfs://hadoop-namenode01/tmp/temp811847645/tmp1546738024"* > 2011-01-31 15:48:41,607 [main] INFO > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed! > 2011-01-31 15:48:41,619 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 1066: Unable to open iterator for alias A > -------------------------------------------------------------------------- > > While running the same LOAD, DUMP works fine with another user account. We > also confirmed there is no diskspace or quota issue on namenode. Any idea? > > This is similar to this issue reported here: > > http://web.archiveorange.com/archive/v/3inw3wuad4S3zjAz89y5 > > >