[jira] [Comment Edited] (YARN-5845) Skip aclUpdated event publish to timelineserver or recovery
[
https://issues.apache.org/jira/browse/YARN-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15644790#comment-15644790
]
Varun Saxena edited comment on YARN-5845 at 11/7/16 5:26 PM:
-
What I meant was report both the events from appCreated implementation and
remove appACLsUpdated method altogether. We do not need to introduce another
method. Required info can be found from RMApp object for both the events.
We can alternatively move the appACLsUpdated call to where appCreated method is
called. That is call them one by one. However I would prefer former as we will
then publish a single entity.
Reporting app ACL event before app is saved in RM (which is currently being
done) may create inconsistency between RM view of app on restart and timeline
view of app (if RM restarts before app is saved in NEW state)
Thoughts ?
was (Author: varun_saxena):
What I meant was report both the events from appCreated implementation and
remove appACLsUpdated method altogether. We do not need to introduce another
method. Required info can be found from RMApp object for both the events.
We can alternatively move the appACLsUpdated call to where appCreated method is
called. That is call them one by one. However I would prefer former as we will
then publish a single entity.
Reporting app ACL event before app is saved in RM (which is currently being
done) may create inconsistency between RM view of app on restart and timeline
view of app (if RM restarts before app is saved in NEW state)
> Skip aclUpdated event publish to timelineserver or recovery
> ---
>
> Key: YARN-5845
> URL: https://issues.apache.org/jira/browse/YARN-5845
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: YARN-5845.0001.patch
>
>
> Currently ACL update event is send to timeline server even on recovery
> {{RMAppManager#createAndPopulateNewRMApp}}.
> For 10K completed application when RM is restarted 10K ACL updated event is
> added to timelinesever causing unnecessary over loading of system
> {code}
> String appViewACLs = submissionContext.getAMContainerSpec()
> .getApplicationACLs().get(ApplicationAccessType.VIEW_APP);
> rmContext.getSystemMetricsPublisher().appACLsUpdated(
> application, appViewACLs, System.currentTimeMillis());
> {code}
> *Events on each RM restart*
> {noformat}
> "events": [{
> "timestamp": 1478520292543,
> "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
> "eventinfo": {}
> }, {
> "timestamp": 1478519600537,
> "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
> "eventinfo": {}
> }, {
> "timestamp": 1478519557101,
> "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
> "eventinfo": {}
> },
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5845) Skip aclUpdated event publish to timelineserver or recovery
[
https://issues.apache.org/jira/browse/YARN-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15644530#comment-15644530
]
Bibin A Chundatt edited comment on YARN-5845 at 11/7/16 3:55 PM:
-
Attaching patch based on solution 1
MInor change in comment from last time instead of startTime should be
*submitTime*. Apologies incase created any confusion
{code}
// call RMAppManager to submit application directly
rmAppManager.submitApplication(submissionContext,
System.currentTimeMillis(), user);
{code}
Application start {{submit Time}} and {{aclUpdate}} time are set to same.
The same appSubmit time will be used to aclUpdate also.
In recovery flow the {{acl update time}} will also same as {{submit Time}} and
no new entries.
was (Author: bibinchundatt):
Attaching patch based on solution 1
MInor comment from last time instead of startTime should be *submitTime*
{code}
// call RMAppManager to submit application directly
rmAppManager.submitApplication(submissionContext,
System.currentTimeMillis(), user);
{code}
Application start {{submit Time}} and {{aclUpdate}} time are set to same.
The same appSubmit time will be used to aclUpdate also.
In recovery flow the {{acl update time}} will also same as {{submit Time}} and
no new entries.
> Skip aclUpdated event publish to timelineserver or recovery
> ---
>
> Key: YARN-5845
> URL: https://issues.apache.org/jira/browse/YARN-5845
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: YARN-5845.0001.patch
>
>
> Currently ACL update event is send to timeline server even on recovery
> {{RMAppManager#createAndPopulateNewRMApp}}.
> For 10K completed application when RM is restarted 10K ACL updated event is
> added to timelinesever causing unnecessary over loading of system
> {code}
> String appViewACLs = submissionContext.getAMContainerSpec()
> .getApplicationACLs().get(ApplicationAccessType.VIEW_APP);
> rmContext.getSystemMetricsPublisher().appACLsUpdated(
> application, appViewACLs, System.currentTimeMillis());
> {code}
> *Events on each RM restart*
> {noformat}
> "events": [{
> "timestamp": 1478520292543,
> "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
> "eventinfo": {}
> }, {
> "timestamp": 1478519600537,
> "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
> "eventinfo": {}
> }, {
> "timestamp": 1478519557101,
> "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
> "eventinfo": {}
> },
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
