[ https://issues.apache.org/jira/browse/YARN-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15933782#comment-15933782 ]
yunjiong zhao commented on YARN-6339: ------------------------------------- {quote}Why changes of createAndGetApplicationReport required? {quote} The purpose is to avoid calling getLogAggregationStatus() unnecessary inside getLogAggregationReportsForApp() after application's LogAggregationStatus changed to TIME_OUT. I think we should add LogAggregationStatus.TIME_OUT in isLogAggregationFinished() like LogAggregationStatus.SUCCEEDED and LogAggregationStatus.FAILED. If ignore future risks, we can even change logAggregationStatusForAppReport inside getLogAggregationStatusForAppReport() with hold readLock only. To avoid confusing, due to createAndGetApplicationReport() will call getLogAggregationStatusForAppReport() with hold readLock, I think update logAggregationStatusForAppReport inside createAndGetApplicationReport() with writeLock hold is right thing to do. {code} } else if (logTimeOutCount > 0) { + logAggregationStatusForAppReport = LogAggregationStatus.TIME_OUT; return LogAggregationStatus.TIME_OUT; } {code} > Improve performance for createAndGetApplicationReport > ----------------------------------------------------- > > Key: YARN-6339 > URL: https://issues.apache.org/jira/browse/YARN-6339 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: yunjiong zhao > Assignee: yunjiong zhao > Attachments: YARN-6339.001.patch, YARN-6339.002.patch > > > There are two performance issue when calling createAndGetApplicationReport: > One is inside ProtoUtils.convertFromProtoFormat, replace is too slow for > clusters which have more than 3000 nodes. Use substring is much better: > https://issues.apache.org/jira/browse/YARN-6285?focusedCommentId=15923241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15923241 > Another one is inside getLogAggregationReportsForApp, if some application's > LogAggregationStatus is TIME_OUT, every time it was called it will create an > HashMap which will produce lots of garbage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org