[ https://issues.apache.org/jira/browse/YARN-4882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781579#comment-15781579 ]
Robert Kanter commented on YARN-4882: ------------------------------------- +1 > Change the log level to DEBUG for recovering completed applications > ------------------------------------------------------------------- > > Key: YARN-4882 > URL: https://issues.apache.org/jira/browse/YARN-4882 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager > Reporter: Rohith Sharma K S > Assignee: Daniel Templeton > Labels: oct16-easy > Attachments: YARN-4882.001.patch, YARN-4882.002.patch, > YARN-4882.003.patch, YARN-4882.004.patch > > > I think for recovering completed applications no need to log as INFO, rather > it can be made it as DEBUG. The problem seen from large cluster is if any > issue happens during RM start up and continuously switching , then RM logs > are filled with most with recovering applications only. > There are 6 lines are logged for 1 applications as I shown in below logs, > then consider RM default value for max-completed applications is 10K. So for > each switch 10K*6=60K lines will be added which is not useful I feel. > {noformat} > 2016-03-01 10:20:59,077 INFO > org.apache.hadoop.yarn.server.resourcemanager.RMAppManager: Default priority > level is set to application:application_1456298208485_21507 > 2016-03-01 10:20:59,094 INFO > org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Recovering > app: application_1456298208485_21507 with 1 attempts and final state = > FINISHED > 2016-03-01 10:20:59,100 INFO > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: > Recovering attempt: appattempt_1456298208485_21507_000001 with final state: > FINISHED > 2016-03-01 10:20:59,107 INFO > org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: > appattempt_1456298208485_21507_000001 State change from NEW to FINISHED > 2016-03-01 10:20:59,111 INFO > org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: > application_1456298208485_21507 State change from NEW to FINISHED > 2016-03-01 10:20:59,112 INFO > org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=rohith > OPERATION=Application Finished - Succeeded TARGET=RMAppManager > RESULT=SUCCESS APPID=application_1456298208485_21507 > {noformat} > The main problem is missing important information's from the logs before RM > unstable. Even though log roll back is 50 or 100, in a short period all these > logs will be rolled out and all the logs contains only RM switching > information that too recovering applications!!. > I suggest at least completed applications recovery should be logged as DEBUG. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org