Batch jobHistory disk flushes
-----------------------------
Key: MAPREDUCE-3512
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3512
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mr-am, mrv2
Affects Versions: 0.23.0
Reporter: Siddharth Seth
The mr-am flushes each individual job history event to disk for AM recovery.
The history even handler ends up with a significant backlog for tests like
MAPREDUCE-3402.
History events could be batched up based on num records / time /
TaskFinishedEvents to reduce the number of DFS writes - with the potential
drawback of having to rerun some tasks during AM recovery.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
