[ https://issues.apache.org/jira/browse/MAPREDUCE-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510899#comment-13510899 ]
Hadoop QA commented on MAPREDUCE-4843: -------------------------------------- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12556081/MAPREDUCE-4843-branch-1.1.patch against trunk revision . {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3095//console This message is automatically generated. > When using DefaultTaskController, JobLocalizer not thread safe > -------------------------------------------------------------- > > Key: MAPREDUCE-4843 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4843 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker > Affects Versions: 1.1.1 > Reporter: zhaoyunjiong > Priority: Critical > Attachments: MAPREDUCE-4843-branch-1.1.patch > > > In our cluster, some times job will failed due to below exception: > 2012-12-03 23:11:54,811 WARN org.apache.hadoop.mapred.TaskTracker: Error > initializing attempt_201212031626_1115_r_000023_0: > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > taskTracker/$username/jobcache/job_201212031626_1115/job.xml in any of the > configured local directories > at > org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:424) > at > org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160) > at > org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1175) > at > org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1058) > at > org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2213) > The root cause is JobLocalizer is not thread safe. > In DefaultTaskController.initializeJob method: > JobLocalizer localizer = new JobLocalizer((JobConf)getConf(), user, > jobid); > but in JobLocalizer, it just simply keep the reference of the conf. > When two TaskLauncher threads(mapLauncher and reduceLauncher) try to > initializeJob at same time, it will have two JobLocalizer, but only one conf > instance. > So some times ttConf.setStrings(JOB_LOCAL_CTXT, localDirs) will reset > previous job's conf. > Then it will cause the previous job's job.xml stored at another user's dir. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira