[
https://issues.apache.org/jira/browse/YARN-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15933782#comment-15933782
]
yunjiong zhao commented on YARN-6339:
-------------------------------------
{quote}Why changes of createAndGetApplicationReport required? {quote}
The purpose is to avoid calling getLogAggregationStatus() unnecessary inside
getLogAggregationReportsForApp() after application's LogAggregationStatus
changed to TIME_OUT.
I think we should add LogAggregationStatus.TIME_OUT in
isLogAggregationFinished() like LogAggregationStatus.SUCCEEDED and
LogAggregationStatus.FAILED.
If ignore future risks, we can even change logAggregationStatusForAppReport
inside getLogAggregationStatusForAppReport() with hold readLock only. To avoid
confusing, due to createAndGetApplicationReport() will call
getLogAggregationStatusForAppReport() with hold readLock, I think update
logAggregationStatusForAppReport inside createAndGetApplicationReport() with
writeLock hold is right thing to do.
{code}
} else if (logTimeOutCount > 0) {
+ logAggregationStatusForAppReport = LogAggregationStatus.TIME_OUT;
return LogAggregationStatus.TIME_OUT;
}
{code}
> Improve performance for createAndGetApplicationReport
> -----------------------------------------------------
>
> Key: YARN-6339
> URL: https://issues.apache.org/jira/browse/YARN-6339
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: yunjiong zhao
> Assignee: yunjiong zhao
> Attachments: YARN-6339.001.patch, YARN-6339.002.patch
>
>
> There are two performance issue when calling createAndGetApplicationReport:
> One is inside ProtoUtils.convertFromProtoFormat, replace is too slow for
> clusters which have more than 3000 nodes. Use substring is much better:
> https://issues.apache.org/jira/browse/YARN-6285?focusedCommentId=15923241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15923241
> Another one is inside getLogAggregationReportsForApp, if some application's
> LogAggregationStatus is TIME_OUT, every time it was called it will create an
> HashMap which will produce lots of garbage.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
