[
https://issues.apache.org/jira/browse/HADOOP-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555287#action_12555287
]
Amareshwari Sri Ramadasu commented on HADOOP-2116:
--------------------------------------------------
In the current state of art, jobCacheDir is
"mapred/local/taskTracker/jobcache/<job_id>/work".
I far as I understood, this needs to be accessible as "job.local.dir", a
job-specific shared directory for use as scratch space.
bq. So, mapred.jar, jobCacheDir, and job.local.dir all need to be different
locations.
here, jobCacheDir (existing) would become job.local.dir now.
If you want jobCacheDir to point to
"mapred/local/taskTracker/jobcache/<job_id>/", this cannot be available via a
config variable. Because it cannot take a unique value as it can be present in
more than one disk. For example, we can have task directory (
mapred/local/taskTracker/jobcache/<job_id>/<taskid>) on a disk otherthan
job.local.dir.
Finally, we will have mapred.jar and job.local.dir (earlier jobCachedir) , both
at different locations.
Thoughts?
> Job.local.dir to be exposed to tasks
> ------------------------------------
>
> Key: HADOOP-2116
> URL: https://issues.apache.org/jira/browse/HADOOP-2116
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.3
> Environment: All
> Reporter: Milind Bhandarkar
> Fix For: 0.16.0
>
>
> Currently, since all task cwds are created under a jobcache directory, users
> that need a job-specific shared directory for use as scratch space, create
> ../work. This is hacky, and will break when HADOOP-2115 is addressed. For
> such jobs, hadoop mapred should expose job.local.dir via localized
> configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
