[ https://issues.apache.org/jira/browse/YARN-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332694#comment-16332694 ]
Carlo Curino commented on YARN-6648: ------------------------------------ [~botong] thanks for the updated patch, I think it is nicer to have them combined (easier to follow). Here a few questions/suggestions (some pretty minor, some more important): # in {{MemoryFederationStateStore.setSubClusterLastHeartbeat}} why do you go through {{getSubcluster}} instead of just doing {{membership.get(subClusterId).setLastHeartBeat(longHeartBeat)}} ? # In {{GPGUtils}} consider using {{DurationFormatUtils.formatDuration(long, string_format)}}, instead of the code you have. # In {{GlobalPolicyGenerator}} ## should we keep the string constants here, or have them in {{YarnConfiguration}} or other places where those are usually defined? ## Is the {{SubClusterCleanerService}} required by every Federation deployment, or is it something we might want to make configurable (runs only if turned on). More generally, should we have a generic mechanism to "start services" in the GPG? # In {{SubClusterCleaner}} ## line 77, is there a way for us to "check" whether the format in the {{StateStore}} is local or UTC? Related is the code around line 100, you seem to doubt the format, and be conservative about it, which might mean the clean-up is at times could be delayed by many hours. Anything better than assuming things and/or being overly conservative? ## In {{SubClusterCleaner}} line 87, maybe a bit verbose? Should some of this be {{LOG.debug}} instead (if so, wrap it in the usual {{if(debugEnabled)}} check)? ## What do you do in case the subCluster {{isUnusable()}}? #In \{{SubClusterCleanerService }} ## type in Javadoc GPE ## I assume we will have many similar "actions run on a schedule", can you make this class more generic (templatize it, so we can re-use it)? ## If the threads crashes, do we have something that restarts it? I see it throws {{Exception}}, anyone restarting the service if it throws? > [GPG] Add SubClusterCleaner in Global Policy Generator > ------------------------------------------------------ > > Key: YARN-6648 > URL: https://issues.apache.org/jira/browse/YARN-6648 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Botong Huang > Assignee: Botong Huang > Priority: Minor > Labels: federation, gpg > Attachments: YARN-6648-YARN-2915.v1.patch, > YARN-6648-YARN-7402.v2.patch, YARN-6648-YARN-7402.v3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org