[
https://issues.apache.org/jira/browse/HADOOP-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558045#action_12558045
]
Devaraj Das commented on HADOOP-2116:
-------------------------------------
The only problem with this are the incompatible changes (like ../work and
../work/script); code, especially scripts that assume paths will break. So, is
everyone okay with this for 0.16? Should we do the symlink stuff to maintain
backward compatibility. As an aside, in the directory organization Owen
suggested, one thing that needs to be added is the common scratch space for all
tasks (like the file cache).
Another thing IMO is that we should probably just do the basic dir organization
as was proposed by Amareshwari earlier and the streaming fix. The magnitude of
the change required by the dir organization proposed by Owen seems pretty
significant and seems aggressive for 0.16. Maybe we can do the remaining for
0.17. Thoughts?
> Job.local.dir to be exposed to tasks
> ------------------------------------
>
> Key: HADOOP-2116
> URL: https://issues.apache.org/jira/browse/HADOOP-2116
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.3
> Environment: All
> Reporter: Milind Bhandarkar
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-2116.txt, patch-2116.txt
>
>
> Currently, since all task cwds are created under a jobcache directory, users
> that need a job-specific shared directory for use as scratch space, create
> ../work. This is hacky, and will break when HADOOP-2115 is addressed. For
> such jobs, hadoop mapred should expose job.local.dir via localized
> configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
