Rohith Sharma K S created YARN-1005: ---------------------------------------
Summary: Log aggregators should check for FSDataOutputStream close before renaming to aggregated file. Key: YARN-1005 URL: https://issues.apache.org/jira/browse/YARN-1005 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 2.0.5-alpha Reporter: Rohith Sharma K S If AggregatedLogFormat.LogWriter.closeWriter() is interuppted, then "remoteNodeTmpLogFileForApp" is renamed to "remoteNodeLogFileForApp" file. This renamed file does not contain valid aggregated logs. There can be situation renamed file can be not in BCFile format. This cause issue while viewing from JobHistoryServer web page. {noformat} 2013-07-27 18:51:14,787 ERROR org.apache.hadoop.yarn.webapp.View: Error getting logs for job_1374918614757_0002 java.io.IOException: Not a valid BCFile. at org.apache.hadoop.io.file.tfile.BCFile$Magic.readAndVerify(BCFile.java:927) at org.apache.hadoop.io.file.tfile.BCFile$Reader.<init>(BCFile.java:628) at org.apache.hadoop.io.file.tfile.TFile$Reader.<init>(TFile.java:804) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.<init>(AggregatedLogFormat.java:337) at org.apache.hadoop.yarn.webapp.log.AggregatedLogsBlock.render(AggregatedLogsBlock.java:89) at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:64) at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74) {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira