[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

2023-09-15 Thread via GitHub 


slfan1989 commented on code in PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#discussion_r1327892936


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java:
##
@@ -362,6 +363,72 @@ public FedAppReportFetcher getFetcher() {
 return fetcher;
   }
 
+  @VisibleForTesting
+  public static void removeApplication(Configuration conf, String 
applicationId)
+  throws Exception {
+FederationStateStoreFacade facade = 
FederationStateStoreFacade.getInstance(conf);
+ApplicationId removeAppId = ApplicationId.fromString(applicationId);
+LOG.info("Deleting application {} from state store.", removeAppId);
+facade.deleteApplicationHomeSubCluster(removeAppId);
+LOG.info("Application is deleted from state store");
+  }
+
+  private static void handFormatStateStore() {
+// TODO: YARN-11548. [Federation] Router Supports Format 
FederationStateStore.
+System.err.println("format-state-store is not yet supported.");
+  }
+
+  private static void handRemoveApplicationFromStateStore(Configuration conf,
+  String applicationId) {
+try {
+  removeApplication(conf, applicationId);
+  System.out.println("Application " + applicationId + " is deleted from 
state store");
+} catch (Exception e) {
+  System.err.println("Application " + applicationId + " error, exception = 
" + e);
+}
+  }
+
+  private static void executeRouterCommand(Configuration conf, String[] args) {
+Options opts = new Options();
+Option formatStateStoreOpt = new Option("format-state-store",  false,
+" Formats the FederationStateStore. " +
+"This will clear the FederationStateStore and " +
+"is useful if past applications are no longer needed. " +
+"This should be run only when the Router is not running.");
+Option removeApplicationFromStateStoreOpt = new 
Option("remove-application-from-state-store",
+false, " Remove the application from FederationStateStore. " +
+ " This should be run only when the Router is not running. ");
+opts.addOption(formatStateStoreOpt);
+opts.addOption(removeApplicationFromStateStoreOpt);
+
+String cmd = args[0];
+
+CommandLine cliParser = null;

Review Comment:
   Thanks for your suggestions! I will improve this part of the code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

2023-09-15 Thread via GitHub 


slfan1989 commented on code in PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#discussion_r1327056638


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/utils/FederationStateStoreFacade.java:
##
@@ -1088,4 +1089,21 @@ public ApplicationSubmissionContext 
getApplicationSubmissionContext(ApplicationI
   public FederationCache getFederationCache() {
 return federationCache;
   }
+
+  /**
+   * Delete the mapping of home {@code SubClusterId} of a previously submitted
+   * {@code ApplicationId}. Currently response is empty if the operation was
+   * successful, if not an exception reporting reason for a failure.
+   *
+   * @param appId ApplicationId.
+   */
+  public void deleteApplicationHomeSubCluster(ApplicationId appId) {
+try {
+  DeleteApplicationHomeSubClusterRequest request =
+  DeleteApplicationHomeSubClusterRequest.newInstance(appId);
+  stateStore.deleteApplicationHomeSubCluster(request);
+} catch (Exception e) {
+  LOG.error("deleteApplicationHomeSubCluster error, applicationId = {}.", 
appId, e);

Review Comment:
   Thank you for your suggestion! I will handle exceptions in the code calling 
this function.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

2023-09-13 Thread via GitHub 


slfan1989 commented on code in PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#discussion_r1325251620


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java:
##
@@ -291,22 +296,30 @@ public static String getProxyHostAndPort(Configuration 
conf) {
   public static void main(String[] argv) {
 Configuration conf = new YarnConfiguration();
 Thread
-.setDefaultUncaughtExceptionHandler(new 
YarnUncaughtExceptionHandler());
+.setDefaultUncaughtExceptionHandler(new 
YarnUncaughtExceptionHandler());
 StringUtils.startupShutdownMessage(Router.class, argv, LOG);
 Router router = new Router();
 try {
-
-  // Remove the old hook if we are rebooting.
-  if (null != routerShutdownHook) {
-ShutdownHookManager.get().removeShutdownHook(routerShutdownHook);
+  GenericOptionsParser hParser = new GenericOptionsParser(conf, argv);
+  argv = hParser.getRemainingArgs();
+  if (argv.length > 1) {

Review Comment:
   Thanks for your suggestion! I will improve this part of the code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

2023-09-13 Thread via GitHub 


slfan1989 commented on code in PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#discussion_r1324441005


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java:
##
@@ -36,13 +37,17 @@
 import org.apache.hadoop.security.SecurityUtil;
 import org.apache.hadoop.security.UserGroupInformation;
 import org.apache.hadoop.service.CompositeService;
-import org.apache.hadoop.util.JvmPauseMonitor;
-import org.apache.hadoop.util.ShutdownHookManager;
-import org.apache.hadoop.util.StringUtils;
-import org.apache.hadoop.util.VersionInfo;
+import org.apache.hadoop.util.*;

Review Comment:
   Thank you very much for your help in reviewing the code! I will improve this 
part of the code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

2023-09-13 Thread via GitHub 


slfan1989 commented on code in PR #6055:
URL: https://github.com/apache/hadoop/pull/6055#discussion_r1324441622


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/Router.java:
##
@@ -291,22 +296,30 @@ public static String getProxyHostAndPort(Configuration 
conf) {
   public static void main(String[] argv) {
 Configuration conf = new YarnConfiguration();
 Thread

Review Comment:
   I will fix it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org