[ https://issues.apache.org/jira/browse/MAPREDUCE-3512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3512: ------------------------------------- Priority: Critical (was: Major) > Batch jobHistory disk flushes > ----------------------------- > > Key: MAPREDUCE-3512 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3512 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: mr-am, mrv2 > Affects Versions: 0.23.0 > Reporter: Siddharth Seth > Priority: Critical > > The mr-am flushes each individual job history event to disk for AM recovery. > The history even handler ends up with a significant backlog for tests like > MAPREDUCE-3402. > History events could be batched up based on num records / time / > TaskFinishedEvents to reduce the number of DFS writes - with the potential > drawback of having to rerun some tasks during AM recovery. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira