[
https://issues.apache.org/jira/browse/MAPREDUCE-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated MAPREDUCE-4517:
----------------------------------
Attachment: MAPREDUCE-4517.patch
Thanks for the review, Tom. I modified the logging to only log a before and
after for the entire heartbeat (and only if something changed). Also
abbreviated field names in the log message so it's not such a long line.
I moved out the headroom logging and log only in debug since that's likely to
change often while the job is running and it often isn't interesting when it
does. It's usually interesting when the AM isn't preempting reducers to make
room for maps, and we can always turn on debug via the logLevel webpage to
dynamically enable the logging of the headroom to figure things out.
> Too many INFO messages written out during AM to RM heartbeat
> ------------------------------------------------------------
>
> Key: MAPREDUCE-4517
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4517
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
> Affects Versions: 0.23.1, 2.0.1-alpha
> Reporter: James Kinley
> Assignee: Jason Lowe
> Priority: Minor
> Labels: patch
> Attachments: MAPREDUCE-4517.patch, MAPREDUCE-4517.patch,
> MAPREDUCE-4517.patch, MAPREDUCE-4517.patch, MAPREDUCE-4517.patch
>
>
> Too many INFO log messages written out during AM to RM heartbeat. Based on
> default frequency of 1000ms (scheduler.heartbeat.interval-ms) either 2 or 4
> INFO messages are written out per second:
> LOG.info("Before Scheduling: " + getStat());
> List<Container> allocatedContainers = getResources();
> LOG.info("After Scheduling: " + getStat());
> if (allocatedContainers.size() > 0) {
> LOG.info("Before Assign: " + getStat());
> scheduledRequests.assign(allocatedContainers);
> LOG.info("After Assign: " + getStat());
> }
> These should probably be changed to DEBUG message to save the log growing too
> quickly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
