[
https://issues.apache.org/jira/browse/MAPREDUCE-5264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rémy SAISSY updated MAPREDUCE-5264:
-----------------------------------
Target Version/s: 3.0.0, 2.1.0-beta
Status: Patch Available (was: Open)
Hi,
in JobHistoryEverntHandler and HistoryFileManager, the
FileAlreadyExistsException was only tracked for Logging purpose.
YarnChild.java::configureLocalDirs() tracked the FileAlreadyExistsException to
determine which task should open the directory for write and which one for read.
<code>
workDir = lDirAlloc.getLocalPathForWrite("work", job);
FileSystem lfs = FileSystem.getLocal(job).getRaw();
boolean madeDir = false;
try {
madeDir = lfs.mkdirs(workDir);
} catch (FileAlreadyExistsException e) {
madeDir = true;
workDir = lDirAlloc.getLocalPathToRead("work", job);
}
<code>
However the method's goal according to its documentation is to create the
mapred-local output directory. Shouldn't it be writable anyway?
I have removed the try catch and the read only open in the patch.
No need to modify or add any test case.
> FileAlreadyExistsException is assumed to be thrown by FileSystem#mkdirs or
> FileContext#mkdir in the codebase
> ------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5264
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5264
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 3.0.0, 2.1.0-beta
> Reporter: Rémy SAISSY
>
> According to https://issues.apache.org/jira/browse/HADOOP-9438,
> FileSystem#mkdirs and FileContext#mkdir do not throw
> FileAlreadyExistsException if the directory already exist.
> Some places in the mapreduce codebase assumes FileSystem#mkdirs or
> FileContext#mkdir throw FileAlreadyExistsException.
> At least the following files are concerned:
> - YarnChild.java
> - JobHistoryEverntHandler.java
> - HistoryFileManager.java
> It would be good to re-review and patch this if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
