[
https://issues.apache.org/jira/browse/MAPREDUCE-5679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846713#comment-13846713
]
Jason Lowe commented on MAPREDUCE-5679:
---------------------------------------
Thanks for the patch, [~liangly]. Patch looks good overall. Would like to see
the code properly indented within the synchronized blocks to match the Hadoop
coding guidelines.
Another minor nit is that the code is holding the locks longer than necessary
(only need to lock fileinfo until data is parsed) but not a must-fix since it's
just test code.
> TestJobHistoryParsing has race condition
> ----------------------------------------
>
> Key: MAPREDUCE-5679
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5679
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Liyin Liang
> Assignee: Liyin Liang
> Attachments: MAPREDUCE-5679-2.diff, MAPREDUCE-5679.diff
>
>
> org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing can fail because of
> race condition.
> {noformat}
> testHistoryParsingWithParseErrors(org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing)
> Time elapsed: 4.102 sec <<< ERROR!
> java.io.IOException: Unable to initialize History Viewer
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:520)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:398)
> at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:137)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:798)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.<init>(JobHistoryParser.java:86)
> at
> org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.<init>(HistoryViewer.java:85)
> at
> org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing.checkHistoryParsing(TestJobHistoryParsing.java:339)
> at
> org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing.testHistoryParsingWithParseErrors(TestJobHistoryParsing.java:125)
> {noformat}
> In the checkHistoryParsing() function, after
> {code}
> HistoryFileInfo fileInfo = jobHistory.getJobFileInfo(jobId);
> {code}
> a thread named MoveIntermediateToDone will be launched to move history file
> from done_intermediate to done directory.
> If the history file is moved,
> {code}
> HistoryViewer viewer = new HistoryViewer(fc.makeQualified(
> fileInfo.getHistoryFile()).toString(), conf, true);
> {code}
> will throw IOException，because the history file is not found.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
