[
https://issues.apache.org/jira/browse/YARN-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332694#comment-16332694
]
Carlo Curino edited comment on YARN-6648 at 1/19/18 6:37 PM:
-------------------------------------------------------------
[~botong] thanks for the updated patch, I think it is nicer to have them
combined (easier to follow).
Here a few questions/suggestions (some pretty minor, some more important):
# in {{MemoryFederationStateStore.setSubClusterLastHeartbeat}} why do you go
through {{getSubcluster}} instead of just doing
{{membership.get(subClusterId).setLastHeartBeat(longHeartBeat)}} ?
# In {{GPGUtils}} consider using {{DurationFormatUtils.formatDuration(long,
string_format)}}, instead of the code you have.
# In {{GlobalPolicyGenerator}}
## should we keep the string constants here, or have them in
{{YarnConfiguration}} or other places where those are usually defined?
## Is the {{SubClusterCleanerService}} required by every Federation
deployment, or is it something we might want to make configurable (runs only if
turned on). More generally, should we have a generic mechanism to "start
services" in the GPG?
# In {{SubClusterCleaner}}
## line 77, is there a way for us to "check" whether the format in the
{{StateStore}} is local or UTC? Related is the code around line 100, you seem
to doubt the format, and be conservative about it, which might mean the
clean-up is at times could be delayed by many hours. Anything better than
assuming things and/or being overly conservative?
## In {{SubClusterCleaner}} line 87, maybe a bit verbose? Should some of this
be {{LOG.debug}} instead (if so, wrap it in the usual {{if(debugEnabled)}}
check)?
## What do you do in case the subCluster {{isUnusable()}}?
# In {{SubClusterCleanerService}}
## type in Javadoc GPE
## I assume we will have many similar "actions run on a schedule", can you
make this class more generic (templatize it, so we can re-use it)?
## If the threads crashes, do we have something that restarts it? I see it
throws {{Exception}}, anyone restarting the service if it throws?
was (Author: curino):
[~botong] thanks for the updated patch, I think it is nicer to have them
combined (easier to follow).
Here a few questions/suggestions (some pretty minor, some more important):
# in {{MemoryFederationStateStore.setSubClusterLastHeartbeat}} why do you go
through {{getSubcluster}} instead of just doing
{{membership.get(subClusterId).setLastHeartBeat(longHeartBeat)}} ?
# In {{GPGUtils}} consider using {{DurationFormatUtils.formatDuration(long,
string_format)}}, instead of the code you have.
# In {{GlobalPolicyGenerator}}
## should we keep the string constants here, or have them in
{{YarnConfiguration}} or other places where those are usually defined?
## Is the {{SubClusterCleanerService}} required by every Federation
deployment, or is it something we might want to make configurable (runs only if
turned on). More generally, should we have a generic mechanism to "start
services" in the GPG?
# In {{SubClusterCleaner}}
## line 77, is there a way for us to "check" whether the format in the
{{StateStore}} is local or UTC? Related is the code around line 100, you seem
to doubt the format, and be conservative about it, which might mean the
clean-up is at times could be delayed by many hours. Anything better than
assuming things and/or being overly conservative?
## In {{SubClusterCleaner}} line 87, maybe a bit verbose? Should some of this
be {{LOG.debug}} instead (if so, wrap it in the usual {{if(debugEnabled)}}
check)?
## What do you do in case the subCluster {{isUnusable()}}?
#In \{{SubClusterCleanerService }}
## type in Javadoc GPE
## I assume we will have many similar "actions run on a schedule", can you
make this class more generic (templatize it, so we can re-use it)?
## If the threads crashes, do we have something that restarts it? I see it
throws {{Exception}}, anyone restarting the service if it throws?
> [GPG] Add SubClusterCleaner in Global Policy Generator
> ------------------------------------------------------
>
> Key: YARN-6648
> URL: https://issues.apache.org/jira/browse/YARN-6648
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Botong Huang
> Assignee: Botong Huang
> Priority: Minor
> Labels: federation, gpg
> Attachments: YARN-6648-YARN-2915.v1.patch,
> YARN-6648-YARN-7402.v2.patch, YARN-6648-YARN-7402.v3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
