[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[ https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=646229&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646229 ] ASF GitHub Bot logged work on HDFS-16210: - Author: ASF GitHub Bot Created on: 03/Sep/21 09:15 Start Date: 03/Sep/21 09:15 Worklog Time Spent: 10m Work Description: symious opened a new pull request #3379: URL: https://github.com/apache/hadoop/pull/3379 ### Description of PR We enabled FairCallQueue to RouterRpcServer, but Router can not refreshCallQueue as NameNode does. This ticket is to enable the refreshCallQueue for Router so that we don't have to restart the Routers when updating FairCallQueue configurations. The option is not to refreshCallQueue to NameNodes, just trying to refresh the callQueue of Router itself. ### How was this patch tested? Unit test ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 646229) Remaining Estimate: 0h Time Spent: 10m > Add the option of refreshCallQueue to RouterAdmin > - > > Key: HDFS-16210 > URL: https://issues.apache.org/jira/browse/HDFS-16210 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > We enabled FairCallQueue to RouterRpcServer, but Router can not > refreshCallQueue as NameNode does. > This ticket is to enable the refreshCallQueue for Router so that we don't > have to restart the Routers when updating FairCallQueue configurations. > > The option is not to refreshCallQueue to NameNodes, just trying to refresh > the callQueue of Router itself. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[ https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=646230&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646230 ] ASF GitHub Bot logged work on HDFS-16210: - Author: ASF GitHub Bot Created on: 03/Sep/21 09:16 Start Date: 03/Sep/21 09:16 Worklog Time Spent: 10m Work Description: symious commented on pull request #3379: URL: https://github.com/apache/hadoop/pull/3379#issuecomment-912390205 @goiri Could you help to review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 646230) Time Spent: 20m (was: 10m) > Add the option of refreshCallQueue to RouterAdmin > - > > Key: HDFS-16210 > URL: https://issues.apache.org/jira/browse/HDFS-16210 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > We enabled FairCallQueue to RouterRpcServer, but Router can not > refreshCallQueue as NameNode does. > This ticket is to enable the refreshCallQueue for Router so that we don't > have to restart the Routers when updating FairCallQueue configurations. > > The option is not to refreshCallQueue to NameNodes, just trying to refresh > the callQueue of Router itself. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[
https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=646258&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646258
]
ASF GitHub Bot logged work on HDFS-16210:
-
Author: ASF GitHub Bot
Created on: 03/Sep/21 11:14
Start Date: 03/Sep/21 11:14
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3379:
URL: https://github.com/apache/hadoop/pull/3379#issuecomment-912460294
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 37m 42s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/1/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 47s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 48s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 54s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 49s | | the patch passed |
| +1 :green_heart: | compile | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 47s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 24s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 4 new + 0
unchanged - 0 fixed = 4 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 44s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 23m 46s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 116m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3379 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 0f01098e0c26 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 75b2b043a1c9b67ef2e9466a5646252a9d6b6dcb |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranc
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[
https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=646335&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646335
]
ASF GitHub Bot logged work on HDFS-16210:
-
Author: ASF GitHub Bot
Created on: 03/Sep/21 14:34
Start Date: 03/Sep/21 14:34
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3379:
URL: https://github.com/apache/hadoop/pull/3379#issuecomment-912586249
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 49s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 41s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 18s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 17s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 20m 4s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 93m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3379 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux daed326725d4 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / de4f45fa8fc6d9b4486654cb2bba0a2afb0624a1 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/2/testReport/ |
| Max. process+thread count | 2634 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[
https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=646797&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646797
]
ASF GitHub Bot logged work on HDFS-16210:
-
Author: ASF GitHub Bot
Created on: 06/Sep/21 08:41
Start Date: 06/Sep/21 08:41
Worklog Time Spent: 10m
Work Description: ferhui commented on a change in pull request #3379:
URL: https://github.com/apache/hadoop/pull/3379#discussion_r702715835
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterAdminCLI.java
##
@@ -1740,6 +1740,15 @@ public void testErrorFaultTolerant() throws Exception {
assertEquals(0, ToolRunner.run(admin, argv));
}
+ @Test
+ public void testRefreshCallQueue() throws Exception {
+
+System.setOut(new PrintStream(out));
+String[] argv = new String[]{"-refreshCallQueue"};
+assertEquals(0, ToolRunner.run(admin, argv));
+assertTrue(out.toString().contains("Refresh call queue successful"));
Review comment:
And here.
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/tools/federation/RouterAdmin.java
##
@@ -1258,6 +1262,39 @@ public int genericRefresh(String[] argv, int i) throws
IOException {
}
}
+ /**
+ * Refresh Router's call Queue.
+ *
+ * @throws IOException if the operation was not successful.
+ */
+ private int refreshCallQueue() throws IOException {
+Configuration conf = getConf();
+String hostport = getConf().getTrimmed(
+RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_KEY,
+RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_DEFAULT);
+
+// Create the client
+Class xface = RefreshCallQueueProtocolPB.class;
+InetSocketAddress address = NetUtils.createSocketAddr(hostport);
+UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
+
+RPC.setProtocolEngine(conf, xface, ProtobufRpcEngine2.class);
+RefreshCallQueueProtocolPB proxy =
(RefreshCallQueueProtocolPB)RPC.getProxy(
+xface, RPC.getProtocolVersion(xface), address, ugi, conf,
+NetUtils.getDefaultSocketFactory(conf), 0);
+
+int returnCode = -1;
+try (RefreshCallQueueProtocolClientSideTranslatorPB xlator =
+new RefreshCallQueueProtocolClientSideTranslatorPB(proxy)) {
+ xlator.refreshCallQueue();
+ System.out.println("Refresh call queue successful for " + hostport);
Review comment:
here successfully？the bellow should be unsuccessfully?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 646797)
Time Spent: 50m (was: 40m)
> Add the option of refreshCallQueue to RouterAdmin
> -
>
> Key: HDFS-16210
> URL: https://issues.apache.org/jira/browse/HDFS-16210
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Janus Chow
>Assignee: Janus Chow
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We enabled FairCallQueue to RouterRpcServer, but Router can not
> refreshCallQueue as NameNode does.
> This ticket is to enable the refreshCallQueue for Router so that we don't
> have to restart the Routers when updating FairCallQueue configurations.
>
> The option is not to refreshCallQueue to NameNodes, just trying to refresh
> the callQueue of Router itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[
https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=646799&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646799
]
ASF GitHub Bot logged work on HDFS-16210:
-
Author: ASF GitHub Bot
Created on: 06/Sep/21 08:49
Start Date: 06/Sep/21 08:49
Worklog Time Spent: 10m
Work Description: symious commented on a change in pull request #3379:
URL: https://github.com/apache/hadoop/pull/3379#discussion_r702721915
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/tools/federation/RouterAdmin.java
##
@@ -1258,6 +1262,39 @@ public int genericRefresh(String[] argv, int i) throws
IOException {
}
}
+ /**
+ * Refresh Router's call Queue.
+ *
+ * @throws IOException if the operation was not successful.
+ */
+ private int refreshCallQueue() throws IOException {
+Configuration conf = getConf();
+String hostport = getConf().getTrimmed(
+RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_KEY,
+RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_DEFAULT);
+
+// Create the client
+Class xface = RefreshCallQueueProtocolPB.class;
+InetSocketAddress address = NetUtils.createSocketAddr(hostport);
+UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
+
+RPC.setProtocolEngine(conf, xface, ProtobufRpcEngine2.class);
+RefreshCallQueueProtocolPB proxy =
(RefreshCallQueueProtocolPB)RPC.getProxy(
+xface, RPC.getProtocolVersion(xface), address, ugi, conf,
+NetUtils.getDefaultSocketFactory(conf), 0);
+
+int returnCode = -1;
+try (RefreshCallQueueProtocolClientSideTranslatorPB xlator =
+new RefreshCallQueueProtocolClientSideTranslatorPB(proxy)) {
+ xlator.refreshCallQueue();
+ System.out.println("Refresh call queue successful for " + hostport);
Review comment:
@ferhui Thanks for the review.
Just to be sure, do you mean changing "successful" to "successfully" and
"failed" to "unsuccessfully"?
Borrowed from DFSAdmin, but the changes are good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 646799)
Time Spent: 1h (was: 50m)
> Add the option of refreshCallQueue to RouterAdmin
> -
>
> Key: HDFS-16210
> URL: https://issues.apache.org/jira/browse/HDFS-16210
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Janus Chow
>Assignee: Janus Chow
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> We enabled FairCallQueue to RouterRpcServer, but Router can not
> refreshCallQueue as NameNode does.
> This ticket is to enable the refreshCallQueue for Router so that we don't
> have to restart the Routers when updating FairCallQueue configurations.
>
> The option is not to refreshCallQueue to NameNodes, just trying to refresh
> the callQueue of Router itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[
https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=646801&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646801
]
ASF GitHub Bot logged work on HDFS-16210:
-
Author: ASF GitHub Bot
Created on: 06/Sep/21 09:00
Start Date: 06/Sep/21 09:00
Worklog Time Spent: 10m
Work Description: ferhui commented on a change in pull request #3379:
URL: https://github.com/apache/hadoop/pull/3379#discussion_r702730385
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/tools/federation/RouterAdmin.java
##
@@ -1258,6 +1262,39 @@ public int genericRefresh(String[] argv, int i) throws
IOException {
}
}
+ /**
+ * Refresh Router's call Queue.
+ *
+ * @throws IOException if the operation was not successful.
+ */
+ private int refreshCallQueue() throws IOException {
+Configuration conf = getConf();
+String hostport = getConf().getTrimmed(
+RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_KEY,
+RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_DEFAULT);
+
+// Create the client
+Class xface = RefreshCallQueueProtocolPB.class;
+InetSocketAddress address = NetUtils.createSocketAddr(hostport);
+UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
+
+RPC.setProtocolEngine(conf, xface, ProtobufRpcEngine2.class);
+RefreshCallQueueProtocolPB proxy =
(RefreshCallQueueProtocolPB)RPC.getProxy(
+xface, RPC.getProtocolVersion(xface), address, ugi, conf,
+NetUtils.getDefaultSocketFactory(conf), 0);
+
+int returnCode = -1;
+try (RefreshCallQueueProtocolClientSideTranslatorPB xlator =
+new RefreshCallQueueProtocolClientSideTranslatorPB(proxy)) {
+ xlator.refreshCallQueue();
+ System.out.println("Refresh call queue successful for " + hostport);
Review comment:
Uh, it's not my native language. Maybe adverb is more suitable here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 646801)
Time Spent: 1h 10m (was: 1h)
> Add the option of refreshCallQueue to RouterAdmin
> -
>
> Key: HDFS-16210
> URL: https://issues.apache.org/jira/browse/HDFS-16210
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Janus Chow
>Assignee: Janus Chow
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> We enabled FairCallQueue to RouterRpcServer, but Router can not
> refreshCallQueue as NameNode does.
> This ticket is to enable the refreshCallQueue for Router so that we don't
> have to restart the Routers when updating FairCallQueue configurations.
>
> The option is not to refreshCallQueue to NameNodes, just trying to refresh
> the callQueue of Router itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[
https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=646815&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646815
]
ASF GitHub Bot logged work on HDFS-16210:
-
Author: ASF GitHub Bot
Created on: 06/Sep/21 09:29
Start Date: 06/Sep/21 09:29
Worklog Time Spent: 10m
Work Description: symious commented on a change in pull request #3379:
URL: https://github.com/apache/hadoop/pull/3379#discussion_r702751678
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/tools/federation/RouterAdmin.java
##
@@ -1258,6 +1262,39 @@ public int genericRefresh(String[] argv, int i) throws
IOException {
}
}
+ /**
+ * Refresh Router's call Queue.
+ *
+ * @throws IOException if the operation was not successful.
+ */
+ private int refreshCallQueue() throws IOException {
+Configuration conf = getConf();
+String hostport = getConf().getTrimmed(
+RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_KEY,
+RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_DEFAULT);
+
+// Create the client
+Class xface = RefreshCallQueueProtocolPB.class;
+InetSocketAddress address = NetUtils.createSocketAddr(hostport);
+UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
+
+RPC.setProtocolEngine(conf, xface, ProtobufRpcEngine2.class);
+RefreshCallQueueProtocolPB proxy =
(RefreshCallQueueProtocolPB)RPC.getProxy(
+xface, RPC.getProtocolVersion(xface), address, ugi, conf,
+NetUtils.getDefaultSocketFactory(conf), 0);
+
+int returnCode = -1;
+try (RefreshCallQueueProtocolClientSideTranslatorPB xlator =
+new RefreshCallQueueProtocolClientSideTranslatorPB(proxy)) {
+ xlator.refreshCallQueue();
+ System.out.println("Refresh call queue successful for " + hostport);
Review comment:
Sure, updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 646815)
Time Spent: 1h 20m (was: 1h 10m)
> Add the option of refreshCallQueue to RouterAdmin
> -
>
> Key: HDFS-16210
> URL: https://issues.apache.org/jira/browse/HDFS-16210
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Janus Chow
>Assignee: Janus Chow
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> We enabled FairCallQueue to RouterRpcServer, but Router can not
> refreshCallQueue as NameNode does.
> This ticket is to enable the refreshCallQueue for Router so that we don't
> have to restart the Routers when updating FairCallQueue configurations.
>
> The option is not to refreshCallQueue to NameNodes, just trying to refresh
> the callQueue of Router itself.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[
https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=647199&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647199
]
ASF GitHub Bot logged work on HDFS-16210:
-
Author: ASF GitHub Bot
Created on: 07/Sep/21 07:12
Start Date: 07/Sep/21 07:12
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3379:
URL: https://github.com/apache/hadoop/pull/3379#issuecomment-914051141
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 40s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 52s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 47s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 41s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 20m 25s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 95m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3379 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux fa3e9f00b16d 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3ade7622250b39d226330984a02e865dd27bc2aa |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/3/testReport/ |
| Max. process+thread count | 2603 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3379/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
[jira] [Work logged] (HDFS-16210) Add the option of refreshCallQueue to RouterAdmin
[ https://issues.apache.org/jira/browse/HDFS-16210?focusedWorklogId=647287&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647287 ] ASF GitHub Bot logged work on HDFS-16210: - Author: ASF GitHub Bot Created on: 07/Sep/21 10:47 Start Date: 07/Sep/21 10:47 Worklog Time Spent: 10m Work Description: ferhui commented on pull request #3379: URL: https://github.com/apache/hadoop/pull/3379#issuecomment-914199242 @symious Could you please change the title of PR and JIRA to "RBF: Add xxx", it means that it's for RBF and it convenient for others to track RBF -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 647287) Time Spent: 1h 40m (was: 1.5h) > Add the option of refreshCallQueue to RouterAdmin > - > > Key: HDFS-16210 > URL: https://issues.apache.org/jira/browse/HDFS-16210 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > We enabled FairCallQueue to RouterRpcServer, but Router can not > refreshCallQueue as NameNode does. > This ticket is to enable the refreshCallQueue for Router so that we don't > have to restart the Routers when updating FairCallQueue configurations. > > The option is not to refreshCallQueue to NameNodes, just trying to refresh > the callQueue of Router itself. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
