[ https://issues.apache.org/jira/browse/MAPREDUCE-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256607#comment-15256607 ]
Ming Ma commented on MAPREDUCE-6315: ------------------------------------ Thanks [~jira.shegalov]! We definitely need to support this scenario. There was some discussion about providing clean-up functionality in YARN-2261 and MAPREDUCE-4428 so that history files can be moved to final location properly. But it isn't clear when we plan to provide such functionality at YARN layer and it seems like a larger effort. The patch here is more targeted and can take care of the issue we have had, at least until the end-to-end clean-up functionality is available. What do you think? Specific comments for the jira: * This will enable "mapred job -logs" usage. How about the "jobhistory URL http://...../jobhistory/job/job_ returns job not found" scenario, is it easy to add the redirect at that level? * globStatus might return empty list. So it might be better to change from {{if (jhStats != null) }} to something like {{if (jhStats != null && jhStats.length > 0)}}. * Is the output format change in JobHistoryParser required? Wonder if there is any backward compatibility issue if some tools have assumption about this. > Implement retrieval of logs for crashed MR-AM via jhist in the staging > directory > -------------------------------------------------------------------------------- > > Key: MAPREDUCE-6315 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6315 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client, mr-am > Affects Versions: 2.7.0 > Reporter: Gera Shegalov > Assignee: Gera Shegalov > Priority: Critical > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6315.001.patch, MAPREDUCE-6315.002.patch, > MAPREDUCE-6315.003.patch > > > When all AM attempts crash, there is no record of them in JHS. Thus no easy > way to get the logs. This JIRA automates the procedure by utilizing the jhist > file in the staging directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)