[ https://issues.apache.org/jira/browse/HADOOP-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12557149#action_12557149 ]
Amareshwari Sri Ramadasu commented on HADOOP-2116: -------------------------------------------------- I propose the following the job cache directory structure to address the above needs: {noformat} mapred/local/tasktracker/jobcache/<jobid>/ --------> job_jar_xml/ ---------> job.jar ---------> job.xml ---------> unJarred directory --------> work/ --------><taskdir> {noformat} And we can have the directories job_jar_xml, work and taskdir on different disks. mapred/local/tasktracker/jobcache/<jobid>/job_jar_xml/job.jar is available via mapred.jar and mapred/local/tasktracker/jobcache/<jobid>/work is available via job.local.dir , which is an empty directory. Thoughts? > Job.local.dir to be exposed to tasks > ------------------------------------ > > Key: HADOOP-2116 > URL: https://issues.apache.org/jira/browse/HADOOP-2116 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.14.3 > Environment: All > Reporter: Milind Bhandarkar > Assignee: Amareshwari Sri Ramadasu > Fix For: 0.16.0 > > > Currently, since all task cwds are created under a jobcache directory, users > that need a job-specific shared directory for use as scratch space, create > ../work. This is hacky, and will break when HADOOP-2115 is addressed. For > such jobs, hadoop mapred should expose job.local.dir via localized > configuration. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.