[GitHub] [hadoop] fengnanli commented on a change in pull request #2266: [RBF] HDFS-15554 Force router check file existence in destinations before adding/updating mount points
fengnanli commented on a change in pull request #2266: URL: https://github.com/apache/hadoop/pull/2266#discussion_r484569373 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterAdminServer.java ## @@ -562,11 +595,35 @@ public GetDestinationResponse getDestination( LOG.error("Cannot get location for {}: {}", src, ioe.getMessage()); } -if (nsIds.isEmpty() && !locations.isEmpty()) { - String nsId = locations.get(0).getNameserviceId(); - nsIds.add(nsId); +return nsIds; + } + + /** + * Verify the file exists in destination nameservices to avoid dangling + * mount points. + * + * @param entry the new mount points added, could be from add or update. + * @return destination nameservices where the file doesn't exist. + * @throws IOException + */ + private List verifyFileInDestinations(MountTable entry) Review comment: Had a discussion with @ayushtkn in the jira ticket and he suggested just making the config to preserve the current workflow and enable this feature optionally. I updated the diff as that and it is not necessary to fix a lot of tests in that case. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] fengnanli commented on a change in pull request #2266: [RBF] HDFS-15554 Force router check file existence in destinations before adding/updating mount points
fengnanli commented on a change in pull request #2266: URL: https://github.com/apache/hadoop/pull/2266#discussion_r483767772 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterAdminServer.java ## @@ -562,11 +595,35 @@ public GetDestinationResponse getDestination( LOG.error("Cannot get location for {}: {}", src, ioe.getMessage()); } -if (nsIds.isEmpty() && !locations.isEmpty()) { - String nsId = locations.get(0).getNameserviceId(); - nsIds.add(nsId); +return nsIds; + } + + /** + * Verify the file exists in destination nameservices to avoid dangling + * mount points. + * + * @param entry the new mount points added, could be from add or update. + * @return destination nameservices where the file doesn't exist. + * @throws IOException + */ + private List verifyFileInDestinations(MountTable entry) Review comment: If that's the case, I will try to fix all in one batch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] fengnanli commented on a change in pull request #2266: [RBF] HDFS-15554 Force router check file existence in destinations before adding/updating mount points
fengnanli commented on a change in pull request #2266: URL: https://github.com/apache/hadoop/pull/2266#discussion_r483399376 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterAdminServer.java ## @@ -562,11 +595,35 @@ public GetDestinationResponse getDestination( LOG.error("Cannot get location for {}: {}", src, ioe.getMessage()); } -if (nsIds.isEmpty() && !locations.isEmpty()) { - String nsId = locations.get(0).getNameserviceId(); - nsIds.add(nsId); +return nsIds; + } + + /** + * Verify the file exists in destination nameservices to avoid dangling + * mount points. + * + * @param entry the new mount points added, could be from add or update. + * @return destination nameservices where the file doesn't exist. + * @throws IOException + */ + private List verifyFileInDestinations(MountTable entry) Review comment: @goiri Uploaded an early version of trying to fix all tests. This is pretty tedious work so before I spend more time on this, let me know your thoughts. There are mainly two types of tests when dealing with mount table: 1. Use mock RouterRpcServer and so on, this way no downstream namenode calls are made. I put the mock as well, see the change for TestRouterAdmin.java 2. Use real downstream namenode interaction, see TestRouterMountTable.java. I created the paths before calling mount points change. I kept thinking a much easier way is to add a Router server side config to turn this on and the default is on. In the tests I can just turn the config off explicitly and this way I don't need to deal with individual tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] fengnanli commented on a change in pull request #2266: [RBF] HDFS-15554 Force router check file existence in destinations before adding/updating mount points
fengnanli commented on a change in pull request #2266: URL: https://github.com/apache/hadoop/pull/2266#discussion_r483126174 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterAdminServer.java ## @@ -562,11 +595,35 @@ public GetDestinationResponse getDestination( LOG.error("Cannot get location for {}: {}", src, ioe.getMessage()); } -if (nsIds.isEmpty() && !locations.isEmpty()) { - String nsId = locations.get(0).getNameserviceId(); - nsIds.add(nsId); +return nsIds; + } + + /** + * Verify the file exists in destination nameservices to avoid dangling + * mount points. + * + * @param entry the new mount points added, could be from add or update. + * @return destination nameservices where the file doesn't exist. + * @throws IOException + */ + private List verifyFileInDestinations(MountTable entry) Review comment: Thanks for the suggestion. I want to involve more people as well since when I started to fix the tests, I found there are quite a few tests targeting/testing cases for dangling mount points. @aajisaka Can you share your thoughts as well? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] fengnanli commented on a change in pull request #2266: [RBF] HDFS-15554 Force router check file existence in destinations before adding/updating mount points
fengnanli commented on a change in pull request #2266: URL: https://github.com/apache/hadoop/pull/2266#discussion_r483126174 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterAdminServer.java ## @@ -562,11 +595,35 @@ public GetDestinationResponse getDestination( LOG.error("Cannot get location for {}: {}", src, ioe.getMessage()); } -if (nsIds.isEmpty() && !locations.isEmpty()) { - String nsId = locations.get(0).getNameserviceId(); - nsIds.add(nsId); +return nsIds; + } + + /** + * Verify the file exists in destination nameservices to avoid dangling + * mount points. + * + * @param entry the new mount points added, could be from add or update. + * @return destination nameservices where the file doesn't exist. + * @throws IOException + */ + private List verifyFileInDestinations(MountTable entry) Review comment: Thanks for the suggestion. I want to involve more people as well since when I started to fix the tests, I found there are quite a few tests targeting the logic of dangling mount points. @aajisaka Can you share your thoughts as well? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] fengnanli commented on a change in pull request #2266: [RBF] HDFS-15554 Force router check file existence in destinations before adding/updating mount points
fengnanli commented on a change in pull request #2266: URL: https://github.com/apache/hadoop/pull/2266#discussion_r481691251 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterAdminServer.java ## @@ -562,11 +595,35 @@ public GetDestinationResponse getDestination( LOG.error("Cannot get location for {}: {}", src, ioe.getMessage()); } -if (nsIds.isEmpty() && !locations.isEmpty()) { - String nsId = locations.get(0).getNameserviceId(); - nsIds.add(nsId); +return nsIds; + } + + /** + * Verify the file exists in destination nameservices to avoid dangling + * mount points. + * + * @param entry the new mount points added, could be from add or update. + * @return destination nameservices where the file doesn't exist. + * @throws IOException + */ + private List verifyFileInDestinations(MountTable entry) Review comment: Yeah, I can add that. This logic change will break a lot of current tests listed by yetus above. What is a good practice here? I have two ideas in mind: 1) make the logic based on a command option when adding/updating mount tables. 2) add mock (if possible) like what I did for TestRouterAdmin.java to fake out the dirs in namenodes. I like 2 since I think this should be the default check, what's your opinion? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] fengnanli commented on a change in pull request #2266: [RBF] HDFS-15554 Force router check file existence in destinations before adding/updating mount points
fengnanli commented on a change in pull request #2266: URL: https://github.com/apache/hadoop/pull/2266#discussion_r481522484 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterAdmin.java ## @@ -103,11 +113,39 @@ public static void globalSetUp() throws Exception { createNamenodeReport("ns1", "nn1", HAServiceState.ACTIVE)); stateStore.refreshCaches(true); +setUpMocks(); + } + + private static void setUpMocks() throws IOException { RouterRpcServer spyRpcServer = Mockito.spy(routerContext.getRouter().createRpcServer()); Whitebox .setInternalState(routerContext.getRouter(), "rpcServer", spyRpcServer); Mockito.doReturn(null).when(spyRpcServer).getFileInfo(Mockito.anyString()); + +mockRpcClient = Mockito.spy(spyRpcServer.getRPCClient()); +Whitebox +.setInternalState(spyRpcServer, "rpcClient", mockRpcClient); +RemoteLocation remoteLocation0 = new RemoteLocation("ns0", "/testdir", null); +RemoteLocation remoteLocation1 = new RemoteLocation("ns1", "/", null); +mockResponse0.put(remoteLocation0, +new HdfsFileStatus.Builder().build()); +Mockito.doReturn(mockResponse0).when(mockRpcClient).invokeConcurrent( +Mockito.eq(Lists.newArrayList(remoteLocation0)), +Mockito.any(RemoteMethod.class), +Mockito.eq(false), +Mockito.eq(false), +Mockito.eq(HdfsFileStatus.class) +); +mockResponse1.put(remoteLocation1, +new HdfsFileStatus.Builder().build()); +Mockito.doReturn(mockResponse1).when(mockRpcClient).invokeConcurrent( Review comment: After adding the new logic in addMount and updateMount, the current tests would fail since they don't have the file created in destination ns. The mock is to avoid the failure for existing tests. I can add a negative test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org