On Thursday 07 August 2008 16:43:10 John Heidemann wrote:
> On Thu, 07 Aug 2008 19:42:05 +0200, "Leon Mergen" wrote:
> >Hello John,
> >
> >On Thu, Aug 7, 2008 at 6:30 PM, John Heidemann <[EMAIL PROTECTED]> wrote:
> >> I have a large Hadoop streaming job that generally works fine,
> >> but a few (2-4) of the ~3000 maps and reduces have problems.
> >> To make matters worse, the problems are system-dependent (we run an a
> >> cluster with machines of slightly different OS versions).
> >> I'd of course like to debug these problems, but they are embedded in a
> >> large job.
> >>
> >> Is there a way to extract the input given to a reducer from a job, given
> >> the task identity?  (This would also be helpful for mappers.)
> >
> >I believe you should set "keep.failed.tasks.files" to true -- this way,
> > give a task id, you can see what input files it has in ~/
> >taskTracker/${taskid}/work (source:
> >http://hadoop.apache.org/core/docs/r0.17.0/mapred_tutorial.html#IsolationR
> >unner )

IsolationRunner does not work as described in the tutorial.  After the task 
hung, I failed it 
via the web interface.  Then I went to the node that was running this task

  $ cd ...local/taskTracker/jobcache/job_200808071645_0001/work
(this path is already different from the tutorial's)

  $ hadoop org.apache.hadoop.mapred.IsolationRunner ../job.xml
Exception in thread "main" java.lang.NullPointerException
        at 
org.apache.hadoop.mapred.IsolationRunner.main(IsolationRunner.java:164)

Looking at IsolationRunner code, I see this:

    164     File workDirName = new File(lDirAlloc.getLocalPathToRead(
    165                                   TaskTracker.getJobCacheSubdir()
    166                                   + Path.SEPARATOR + taskId.getJobID()
    167                                   + Path.SEPARATOR + taskId
    168                                   + Path.SEPARATOR + "work",
    169                                   conf). toString());

I.e. it assumes there is supposed to be a taskID subdirectory under the job
dir, but:
 $ pwd
 ...mapred/local/taskTracker/jobcache/job_200808071645_0001
 $ ls
 jars  job.xml  work

-- it's not there.  Any suggestions?

Thanks,

  -Yuri


Reply via email to