[
https://issues.apache.org/jira/browse/HADOOP-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12557149#action_12557149
]
Amareshwari Sri Ramadasu commented on HADOOP-2116:
--------------------------------------------------
I propose the following the job cache directory structure to address the above
needs:
{noformat}
mapred/local/tasktracker/jobcache/<jobid>/
--------> job_jar_xml/
---------> job.jar
---------> job.xml
---------> unJarred directory
--------> work/
--------><taskdir>
{noformat}
And we can have the directories job_jar_xml, work and taskdir on different
disks.
mapred/local/tasktracker/jobcache/<jobid>/job_jar_xml/job.jar is available via
mapred.jar
and mapred/local/tasktracker/jobcache/<jobid>/work is available via
job.local.dir , which is an empty directory.
Thoughts?
> Job.local.dir to be exposed to tasks
> ------------------------------------
>
> Key: HADOOP-2116
> URL: https://issues.apache.org/jira/browse/HADOOP-2116
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.3
> Environment: All
> Reporter: Milind Bhandarkar
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
>
> Currently, since all task cwds are created under a jobcache directory, users
> that need a job-specific shared directory for use as scratch space, create
> ../work. This is hacky, and will break when HADOOP-2115 is addressed. For
> such jobs, hadoop mapred should expose job.local.dir via localized
> configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
