[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[ https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=648513=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648513 ] ASF GitHub Bot logged work on HDFS-16065: - Author: ASF GitHub Bot Created on: 09/Sep/21 10:45 Start Date: 09/Sep/21 10:45 Worklog Time Spent: 10m Work Description: ferhui commented on pull request #3100: URL: https://github.com/apache/hadoop/pull/3100#issuecomment-915973951 @symious Thanks for contribution. @goiri Thanks for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 648513) Time Spent: 2.5h (was: 2h 20m) > RBF: Add metrics to record Router's operations > -- > > Key: HDFS-16065 > URL: https://issues.apache.org/jira/browse/HDFS-16065 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Reporter: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Currently, Router's operations are not well recorded. It would be good to > have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for > NameNode, which shows the count for each operations. > Besides, some operations are invoked concurrently in Routers, know the counts > for concurrent operations would help us better knowing about the cluster's > state. > This ticket is to add normal operation metrics and concurrent operation > metrics for Router. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[ https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=648512=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648512 ] ASF GitHub Bot logged work on HDFS-16065: - Author: ASF GitHub Bot Created on: 09/Sep/21 10:45 Start Date: 09/Sep/21 10:45 Worklog Time Spent: 10m Work Description: ferhui merged pull request #3100: URL: https://github.com/apache/hadoop/pull/3100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 648512) Time Spent: 2h 20m (was: 2h 10m) > RBF: Add metrics to record Router's operations > -- > > Key: HDFS-16065 > URL: https://issues.apache.org/jira/browse/HDFS-16065 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Reporter: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Currently, Router's operations are not well recorded. It would be good to > have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for > NameNode, which shows the count for each operations. > Besides, some operations are invoked concurrently in Routers, know the counts > for concurrent operations would help us better knowing about the cluster's > state. > This ticket is to add normal operation metrics and concurrent operation > metrics for Router. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[ https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=612113=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-612113 ] ASF GitHub Bot logged work on HDFS-16065: - Author: ASF GitHub Bot Created on: 18/Jun/21 21:21 Start Date: 18/Jun/21 21:21 Worklog Time Spent: 10m Work Description: goiri commented on a change in pull request #3100: URL: https://github.com/apache/hadoop/pull/3100#discussion_r653838543 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientMetrics.java ## @@ -0,0 +1,646 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hdfs.server.federation.router; Review comment: We can ignore those. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 612113) Time Spent: 2h 10m (was: 2h) > RBF: Add metrics to record Router's operations > -- > > Key: HDFS-16065 > URL: https://issues.apache.org/jira/browse/HDFS-16065 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Reporter: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > Currently, Router's operations are not well recorded. It would be good to > have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for > NameNode, which shows the count for each operations. > Besides, some operations are invoked concurrently in Routers, know the counts > for concurrent operations would help us better knowing about the cluster's > state. > This ticket is to add normal operation metrics and concurrent operation > metrics for Router. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[ https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=611922=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-611922 ] ASF GitHub Bot logged work on HDFS-16065: - Author: ASF GitHub Bot Created on: 18/Jun/21 20:57 Start Date: 18/Jun/21 20:57 Worklog Time Spent: 10m Work Description: symious commented on a change in pull request #3100: URL: https://github.com/apache/hadoop/pull/3100#discussion_r654378797 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientMetrics.java ## @@ -0,0 +1,646 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hdfs.server.federation.router; Review comment: Ok, thanks for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 611922) Time Spent: 2h (was: 1h 50m) > RBF: Add metrics to record Router's operations > -- > > Key: HDFS-16065 > URL: https://issues.apache.org/jira/browse/HDFS-16065 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Reporter: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > Currently, Router's operations are not well recorded. It would be good to > have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for > NameNode, which shows the count for each operations. > Besides, some operations are invoked concurrently in Routers, know the counts > for concurrent operations would help us better knowing about the cluster's > state. > This ticket is to add normal operation metrics and concurrent operation > metrics for Router. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[
https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=611788=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-611788
]
ASF GitHub Bot logged work on HDFS-16065:
-
Author: ASF GitHub Bot
Created on: 16/Jun/21 07:47
Start Date: 16/Jun/21 07:47
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3100:
URL: https://github.com/apache/hadoop/pull/3100#issuecomment-862134661
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 58s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 7s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 15m 27s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 21s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3100/5/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 569 new + 0
unchanged - 0 fixed = 569 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 27m 12s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 102m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3100/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3100 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux c3b0a7e76131 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 674d510e7f4ad0f7c74ed0254df608931f061d26 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[
https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=611745=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-611745
]
ASF GitHub Bot logged work on HDFS-16065:
-
Author: ASF GitHub Bot
Created on: 16/Jun/21 06:00
Start Date: 16/Jun/21 06:00
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3100:
URL: https://github.com/apache/hadoop/pull/3100#issuecomment-862067981
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 0s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 35s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 54s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 23s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3100/4/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 568 new + 0
unchanged - 0 fixed = 568 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 49s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| -1 :x: | spotbugs | 1m 20s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3100/4/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.html)
| hadoop-hdfs-project/hadoop-hdfs-rbf generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | shadedclient | 14m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 26m 2s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3100/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 102m 43s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-hdfs-project/hadoop-hdfs-rbf |
| | Dead store to jm in
org.apache.hadoop.hdfs.server.federation.router.RouterClientMetrics.create(Configuration)
At
RouterClientMetrics.java:org.apache.hadoop.hdfs.server.federation.router.RouterClientMetrics.create(Configuration)
At RouterClientMetrics.java:[line 197] |
| Failed junit tests |
hadoop.hdfs.server.federation.security.TestRouterSecurityManager |
| |
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[
https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=611466=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-611466
]
ASF GitHub Bot logged work on HDFS-16065:
-
Author: ASF GitHub Bot
Created on: 15/Jun/21 17:09
Start Date: 15/Jun/21 17:09
Worklog Time Spent: 10m
Work Description: goiri commented on a change in pull request #3100:
URL: https://github.com/apache/hadoop/pull/3100#discussion_r651989668
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -471,6 +471,9 @@ private Object invokeMethod(
if (this.rpcMonitor != null) {
this.rpcMonitor.proxyOpComplete(true);
}
+if (this.router.getRouterMetrics() != null) {
+ this.router.getRouterMetrics().incInvokedMethod(method);
Review comment:
I think it would be better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 611466)
Time Spent: 1.5h (was: 1h 20m)
> RBF: Add metrics to record Router's operations
> --
>
> Key: HDFS-16065
> URL: https://issues.apache.org/jira/browse/HDFS-16065
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
>Reporter: Janus Chow
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Currently, Router's operations are not well recorded. It would be good to
> have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for
> NameNode, which shows the count for each operations.
> Besides, some operations are invoked concurrently in Routers, know the counts
> for concurrent operations would help us better knowing about the cluster's
> state.
> This ticket is to add normal operation metrics and concurrent operation
> metrics for Router.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[
https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=611366=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-611366
]
ASF GitHub Bot logged work on HDFS-16065:
-
Author: ASF GitHub Bot
Created on: 15/Jun/21 14:25
Start Date: 15/Jun/21 14:25
Worklog Time Spent: 10m
Work Description: symious commented on a change in pull request #3100:
URL: https://github.com/apache/hadoop/pull/3100#discussion_r651845997
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -471,6 +471,9 @@ private Object invokeMethod(
if (this.rpcMonitor != null) {
this.rpcMonitor.proxyOpComplete(true);
}
+if (this.router.getRouterMetrics() != null) {
+ this.router.getRouterMetrics().incInvokedMethod(method);
Review comment:
The metrics name is "RouterActivity", seems a little related. Or do we
need to create a new one named "RouterClientActivity"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 611366)
Time Spent: 1h 20m (was: 1h 10m)
> RBF: Add metrics to record Router's operations
> --
>
> Key: HDFS-16065
> URL: https://issues.apache.org/jira/browse/HDFS-16065
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
>Reporter: Janus Chow
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, Router's operations are not well recorded. It would be good to
> have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for
> NameNode, which shows the count for each operations.
> Besides, some operations are invoked concurrently in Routers, know the counts
> for concurrent operations would help us better knowing about the cluster's
> state.
> This ticket is to add normal operation metrics and concurrent operation
> metrics for Router.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[
https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=610758=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-610758
]
ASF GitHub Bot logged work on HDFS-16065:
-
Author: ASF GitHub Bot
Created on: 14/Jun/21 16:26
Start Date: 14/Jun/21 16:26
Worklog Time Spent: 10m
Work Description: goiri commented on a change in pull request #3100:
URL: https://github.com/apache/hadoop/pull/3100#discussion_r651097583
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -471,6 +471,9 @@ private Object invokeMethod(
if (this.rpcMonitor != null) {
this.rpcMonitor.proxyOpComplete(true);
}
+if (this.router.getRouterMetrics() != null) {
+ this.router.getRouterMetrics().incInvokedMethod(method);
Review comment:
You are adding all these metrics raw to RouterMetrics.
I'm wondering if we should have something that refers to this being metrics
for the client.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 610758)
Time Spent: 1h 10m (was: 1h)
> RBF: Add metrics to record Router's operations
> --
>
> Key: HDFS-16065
> URL: https://issues.apache.org/jira/browse/HDFS-16065
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
>Reporter: Janus Chow
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Currently, Router's operations are not well recorded. It would be good to
> have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for
> NameNode, which shows the count for each operations.
> Besides, some operations are invoked concurrently in Routers, know the counts
> for concurrent operations would help us better knowing about the cluster's
> state.
> This ticket is to add normal operation metrics and concurrent operation
> metrics for Router.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[
https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=610744=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-610744
]
ASF GitHub Bot logged work on HDFS-16065:
-
Author: ASF GitHub Bot
Created on: 14/Jun/21 15:51
Start Date: 14/Jun/21 15:51
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3100:
URL: https://github.com/apache/hadoop/pull/3100#issuecomment-860794285
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 52s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 22s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3100/3/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 573 new + 0
unchanged - 0 fixed = 573 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 19s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 98m 29s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3100/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3100 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 638e995dec2b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 00e68678f1ce2a0e21a640774d00bcd1859821d1 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3100/3/testReport/ |
| Max. process+thread count | 2359 (vs. ulimit of 5500) |
| modules | C:
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[
https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=610565=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-610565
]
ASF GitHub Bot logged work on HDFS-16065:
-
Author: ASF GitHub Bot
Created on: 14/Jun/21 08:07
Start Date: 14/Jun/21 08:07
Worklog Time Spent: 10m
Work Description: symious commented on a change in pull request #3100:
URL: https://github.com/apache/hadoop/pull/3100#discussion_r650457056
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -471,6 +471,9 @@ private Object invokeMethod(
if (this.rpcMonitor != null) {
this.rpcMonitor.proxyOpComplete(true);
}
+if (this.router.getRouterMetrics() != null) {
+ this.router.getRouterMetrics().incInvokedMethod(method);
Review comment:
@goiri Thanks for the review.
A similar metrics for NameNode is
"org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics".
I think this RouterMetrics is trying to monitor from a different view of
RouterRpcServer. IMHO, RouterRpcServer is monitoring Router as a Server, the
new metrics would be treating Router as a Client. Some operations are not
triggered by user requests but from Router itself.
In our cluster, Router's async call queue was jammed, but we couldn't get
the operations from the RouterRpcServer metrics. Then we found most of the
operations are "getDatanodeReport" which are invoked by Router's
FederationMetrics and not by users.
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/metrics/TestRouterMetrics.java
##
@@ -0,0 +1,121 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.federation.metrics;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hdfs.DFSConfigKeys;
+import org.apache.hadoop.hdfs.HdfsConfiguration;
+import org.apache.hadoop.hdfs.server.federation.MiniRouterDFSCluster;
+import org.apache.hadoop.hdfs.server.federation.RouterConfigBuilder;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.io.IOException;
+
+import static org.apache.hadoop.test.MetricsAsserts.assertCounter;
+import static org.apache.hadoop.test.MetricsAsserts.getMetrics;
+
+/**
+ * Test case for FilesInGetListingOps metric in Namenode
+ */
+public class TestRouterMetrics {
+ private static final Configuration CONF = new HdfsConfiguration();
+ private static final String ROUTER_METRICS = "RouterActivity";
+ static {
+CONF.setLong(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 100);
+CONF.setInt(DFSConfigKeys.DFS_BYTES_PER_CHECKSUM_KEY, 1);
+CONF.setLong(DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY, 1L);
+CONF.setInt(DFSConfigKeys.DFS_NAMENODE_REDUNDANCY_INTERVAL_SECONDS_KEY, 1);
+ }
+
+ private static final int NUM_SUBCLUSTERS = 2;
+ private static final int NUM_DNS = 3;
+
+
+ /** Federated HDFS cluster. */
+ private static MiniRouterDFSCluster cluster;
+
+ /** Random Router for this federated cluster. */
+ private MiniRouterDFSCluster.RouterContext router;
+
+ /** Filesystem interface to the Router. */
+ private FileSystem routerFS;
+ /** Filesystem interface to the Namenode. */
+ private FileSystem nnFS;
+
+ @BeforeClass
+ public static void globalSetUp() throws Exception {
+cluster = new MiniRouterDFSCluster(false, NUM_SUBCLUSTERS);
+cluster.setNumDatanodesPerNameservice(NUM_DNS);
+cluster.startCluster();
+
+Configuration routerConf = new RouterConfigBuilder()
+.metrics()
+.rpc()
+.build();
+cluster.addRouterOverrides(routerConf);
+cluster.startRouters();
+
+// Register and verify all NNs with all routers
+cluster.registerNamenodes();
+cluster.waitNamenodeRegistration();
+
+ }
+
+ @Before
+ public void testSetup() throws Exception {
+// Create mock locations
+
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[
https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=610483=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-610483
]
ASF GitHub Bot logged work on HDFS-16065:
-
Author: ASF GitHub Bot
Created on: 14/Jun/21 07:56
Start Date: 14/Jun/21 07:56
Worklog Time Spent: 10m
Work Description: goiri commented on a change in pull request #3100:
URL: https://github.com/apache/hadoop/pull/3100#discussion_r65027
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java
##
@@ -471,6 +471,9 @@ private Object invokeMethod(
if (this.rpcMonitor != null) {
this.rpcMonitor.proxyOpComplete(true);
}
+if (this.router.getRouterMetrics() != null) {
+ this.router.getRouterMetrics().incInvokedMethod(method);
Review comment:
Can you point me to the equivalent for the Namenode?
I thought that the RouterRpcServer was already tracking most of these
metrics.
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/metrics/TestRouterMetrics.java
##
@@ -0,0 +1,121 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hdfs.server.federation.metrics;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hdfs.DFSConfigKeys;
+import org.apache.hadoop.hdfs.HdfsConfiguration;
+import org.apache.hadoop.hdfs.server.federation.MiniRouterDFSCluster;
+import org.apache.hadoop.hdfs.server.federation.RouterConfigBuilder;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.io.IOException;
+
+import static org.apache.hadoop.test.MetricsAsserts.assertCounter;
+import static org.apache.hadoop.test.MetricsAsserts.getMetrics;
+
+/**
+ * Test case for FilesInGetListingOps metric in Namenode
+ */
+public class TestRouterMetrics {
+ private static final Configuration CONF = new HdfsConfiguration();
+ private static final String ROUTER_METRICS = "RouterActivity";
+ static {
+CONF.setLong(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 100);
+CONF.setInt(DFSConfigKeys.DFS_BYTES_PER_CHECKSUM_KEY, 1);
+CONF.setLong(DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY, 1L);
+CONF.setInt(DFSConfigKeys.DFS_NAMENODE_REDUNDANCY_INTERVAL_SECONDS_KEY, 1);
+ }
+
+ private static final int NUM_SUBCLUSTERS = 2;
+ private static final int NUM_DNS = 3;
+
+
+ /** Federated HDFS cluster. */
+ private static MiniRouterDFSCluster cluster;
+
+ /** Random Router for this federated cluster. */
+ private MiniRouterDFSCluster.RouterContext router;
+
+ /** Filesystem interface to the Router. */
+ private FileSystem routerFS;
+ /** Filesystem interface to the Namenode. */
+ private FileSystem nnFS;
+
+ @BeforeClass
+ public static void globalSetUp() throws Exception {
+cluster = new MiniRouterDFSCluster(false, NUM_SUBCLUSTERS);
+cluster.setNumDatanodesPerNameservice(NUM_DNS);
+cluster.startCluster();
+
+Configuration routerConf = new RouterConfigBuilder()
+.metrics()
+.rpc()
+.build();
+cluster.addRouterOverrides(routerConf);
+cluster.startRouters();
+
+// Register and verify all NNs with all routers
+cluster.registerNamenodes();
+cluster.waitNamenodeRegistration();
+
+ }
+
+ @Before
+ public void testSetup() throws Exception {
+// Create mock locations
+cluster.installMockLocations();
+
+// Delete all files via the NNs and verify
+cluster.deleteAllFiles();
+
+// Create test fixtures on NN
+cluster.createTestDirectoriesNamenode();
+
+// Wait to ensure NN has fully created its test directories
+Thread.sleep(100);
+
+router = cluster.getRouters().get(0);
+this.routerFS = router.getFileSystem();
+
+ }
+
+ @AfterClass
+ public static void tearDown() throws Exception {
+cluster.shutdown();
+ }
+
+ @Test
+ public void testGetListing() throws IOException {
Review
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[ https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=610356=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-610356 ] ASF GitHub Bot logged work on HDFS-16065: - Author: ASF GitHub Bot Created on: 14/Jun/21 07:42 Start Date: 14/Jun/21 07:42 Worklog Time Spent: 10m Work Description: symious commented on pull request #3100: URL: https://github.com/apache/hadoop/pull/3100#issuecomment-859595826 @goiri Could you have a look at this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 610356) Time Spent: 0.5h (was: 20m) > RBF: Add metrics to record Router's operations > -- > > Key: HDFS-16065 > URL: https://issues.apache.org/jira/browse/HDFS-16065 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Reporter: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently, Router's operations are not well recorded. It would be good to > have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for > NameNode, which shows the count for each operations. > Besides, some operations are invoked concurrently in Routers, know the counts > for concurrent operations would help us better knowing about the cluster's > state. > This ticket is to add normal operation metrics and concurrent operation > metrics for Router. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[ https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=610349=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-610349 ] ASF GitHub Bot logged work on HDFS-16065: - Author: ASF GitHub Bot Created on: 14/Jun/21 07:41 Start Date: 14/Jun/21 07:41 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3100: URL: https://github.com/apache/hadoop/pull/3100#issuecomment-859660072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 610349) Time Spent: 20m (was: 10m) > RBF: Add metrics to record Router's operations > -- > > Key: HDFS-16065 > URL: https://issues.apache.org/jira/browse/HDFS-16065 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Reporter: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Currently, Router's operations are not well recorded. It would be good to > have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for > NameNode, which shows the count for each operations. > Besides, some operations are invoked concurrently in Routers, know the counts > for concurrent operations would help us better knowing about the cluster's > state. > This ticket is to add normal operation metrics and concurrent operation > metrics for Router. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16065) RBF: Add metrics to record Router's operations
[ https://issues.apache.org/jira/browse/HDFS-16065?focusedWorklogId=610341=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-610341 ] ASF GitHub Bot logged work on HDFS-16065: - Author: ASF GitHub Bot Created on: 14/Jun/21 07:40 Start Date: 14/Jun/21 07:40 Worklog Time Spent: 10m Work Description: symious opened a new pull request #3100: URL: https://github.com/apache/hadoop/pull/3100 ## What changes were proposed in this pull request? Currently, Router's operations are not well recorded. It would be good to have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for NameNode, which shows the count for each operations. Besides, some operations are invoked concurrently in Routers, know the counts for concurrent operations would help us better knowing about the cluster's state. This ticket is to add normal operation metrics and concurrent operation metrics for Router. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDFS-16065 ## How was this patch tested? Add unit test to test normal operation and concurrent ooperation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 610341) Remaining Estimate: 0h Time Spent: 10m > RBF: Add metrics to record Router's operations > -- > > Key: HDFS-16065 > URL: https://issues.apache.org/jira/browse/HDFS-16065 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Reporter: Janus Chow >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently, Router's operations are not well recorded. It would be good to > have a similar metrics as "Hadoop:service=NameNode,name=NameNodeActivity" for > NameNode, which shows the count for each operations. > Besides, some operations are invoked concurrently in Routers, know the counts > for concurrent operations would help us better knowing about the cluster's > state. > This ticket is to add normal operation metrics and concurrent operation > metrics for Router. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
