[ https://issues.apache.org/jira/browse/MAPREDUCE-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli reopened MAPREDUCE-3975: ------------------------------------------------ Sorry for coming late on this: bq. I think that it is fine to have them in different base directories, if it becomes a problem in practice because tasks are trying to share data with each other, then it can be fixed then. This is exactly what the job.local.dir is intended for - sharing of data across tasks on the node. If the tasks only need scratch space, they can write in ./tmp directory which is already created by nodemanager. Even our documentation for the 20.**/1.** release lines explicitly says this. We need to fix it. I am reopening this ticket. > Default value not set for Configuration parameter mapreduce.job.local.dir > ------------------------------------------------------------------------- > > Key: MAPREDUCE-3975 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3975 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 > Affects Versions: 0.23.1, 0.23.2 > Reporter: Eric Payne > Assignee: Eric Payne > Priority: Blocker > Fix For: 0.23.2 > > Attachments: MAPREDUCE-3975-1.txt > > > mapreduce.job.local.dir (formerly job.local.dir in 0.20) is not set by > default. This is a regression from 0.20.205. > In 0.20.205, JobLocalizer.createWorkDir() constructs the > "$mapred.local.dir/taskTracker/$user/jobcache/$jobid/work" path based on > $user and $jobid, and then sets TaskTracker.JOB_LOCAL_DIR in the job's > JobConf. > So far, I haven't found where this is done in 0.23. It could be that this is > what should be done by LocalJobRunner.setupChildMapredLocalDirs(), but I am > still investigating. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira