[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667437#comment-13667437
]
Xi Fang commented on MAPREDUCE-5224:
------------------------------------
Thank Ivan for MAPREDUCE-5224.5.patch.
Here is the reason (offline emails from Ivan) for posting this new patch.
1. Given that fs is indeed used on some other places, we have to account for
that as well (these tests actually want to close the system dir fs).
2. There is no need to use the default file system for the jobhistory. There is
another (orthogonal) bug here. Job history completed location also assumes the
default FS what is not correct. This should be a separate Jira.
3. This would make the prod code change really simple.
> JobTracker should allow the system directory to be in non-default FS
> --------------------------------------------------------------------
>
> Key: MAPREDUCE-5224
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5224
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Reporter: Xi Fang
> Assignee: Xi Fang
> Priority: Minor
> Fix For: 1-win
>
> Attachments: MAPREDUCE-5224.2.patch, MAPREDUCE-5224.3.patch,
> MAPREDUCE-5224.4.patch, MAPREDUCE-5224.5.patch, MAPREDUCE-5224.patch
>
>
> JobTracker today expects the system directory to be in the default file
> system
> if (fs == null) {
> fs = mrOwner.doAs(new PrivilegedExceptionAction<FileSystem>() {
> public FileSystem run() throws IOException {
> return FileSystem.get(conf);
> }});
> }
> ...
> public String getSystemDir() {
> Path sysDir = new Path(conf.get("mapred.system.dir",
> "/tmp/hadoop/mapred/system"));
> return fs.makeQualified(sysDir).toString();
> }
> In Cloud like Azure the default file system is set as ASV (Windows Azure Blob
> Storage), but we would still like the system directory to be in DFS. We
> should change JobTracker to allow that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
