[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[
https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575716#comment-17575716
]
ASF GitHub Bot commented on HDFS-16703:
---
hadoop-yetus commented on PR #4660:
URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1206343919
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 11s | | trunk passed |
| +1 :green_heart: | compile | 6m 50s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 6m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 49s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 12s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 47s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 18s | | the patch passed |
| +1 :green_heart: | compile | 6m 39s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 6m 39s | |
hadoop-hdfs-project-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 1134
unchanged - 1 fixed = 1134 total (was 1135) |
| +1 :green_heart: | compile | 6m 15s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 6m 15s | |
hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 1103
unchanged - 1 fixed = 1103 total (was 1104) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 42s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 7s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 29s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 345m 30s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 501m 23s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4660/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4660 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux d70449d3d50f 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0f07908a8f36f476647123a
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[
https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576054#comment-17576054
]
ASF GitHub Bot commented on HDFS-16703:
---
goiri commented on code in PR #4660:
URL: https://github.com/apache/hadoop/pull/4660#discussion_r939242059
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:
##
@@ -2029,5 +2029,33 @@ public class DFSConfigKeys extends
CommonConfigurationKeys {
public static final long DFS_LEASE_HARDLIMIT_DEFAULT =
HdfsClientConfigKeys.DFS_LEASE_HARDLIMIT_DEFAULT;
+ public static final String IPC_RPC_TIMEOUT_FOR_ALIASHMAP_PROTOCOL =
Review Comment:
Actually, this is correct. My bad.
> Enable RPC Timeout for some protocols of NameNode.
> --
>
> Key: HDFS-16703
> URL: https://issues.apache.org/jira/browse/HDFS-16703
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When I read some code about protocol, I found that only
> ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy
> without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB,
> RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB,
> GetUserMappingsProtocolPB and NamenodeProtocolPB.
>
> If proxy without rpc timeout, it will be blocked for a long time if the NN
> machine crash or bad network during writing or reading with NN.
>
> So I feel that we should enable RPC timeout for all ProtocolPB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[
https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576053#comment-17576053
]
ASF GitHub Bot commented on HDFS-16703:
---
goiri commented on code in PR #4660:
URL: https://github.com/apache/hadoop/pull/4660#discussion_r939239255
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:
##
@@ -2029,5 +2029,33 @@ public class DFSConfigKeys extends
CommonConfigurationKeys {
public static final long DFS_LEASE_HARDLIMIT_DEFAULT =
HdfsClientConfigKeys.DFS_LEASE_HARDLIMIT_DEFAULT;
+ public static final String IPC_RPC_TIMEOUT_FOR_ALIASHMAP_PROTOCOL =
Review Comment:
Should this be HDFS configs?
> Enable RPC Timeout for some protocols of NameNode.
> --
>
> Key: HDFS-16703
> URL: https://issues.apache.org/jira/browse/HDFS-16703
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When I read some code about protocol, I found that only
> ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy
> without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB,
> RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB,
> GetUserMappingsProtocolPB and NamenodeProtocolPB.
>
> If proxy without rpc timeout, it will be blocked for a long time if the NN
> machine crash or bad network during writing or reading with NN.
>
> So I feel that we should enable RPC timeout for all ProtocolPB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576088#comment-17576088 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1207092541 @goiri Master, thanks for your helping review. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17578790#comment-17578790 ] ASF GitHub Bot commented on HDFS-16703: --- slfan1989 commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1212723170 @ZanderXu Do we need to update the md file to explain what these parameters do? > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17579630#comment-17579630 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1214869969 @slfan1989 Thanks for your review. > Do we need to update the md file to explain what these parameters do? Do you mean which MD file? `ipc.client.rpc-timeout.ms` is not been explained. If you think we should do it, I will add some explanation for them. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[
https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17579824#comment-17579824
]
ASF GitHub Bot commented on HDFS-16703:
---
hadoop-yetus commented on PR #4660:
URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1215432652
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 37s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 10s | | trunk passed |
| +1 :green_heart: | compile | 7m 4s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 6m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 14s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 53s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 26s | | the patch passed |
| +1 :green_heart: | compile | 6m 45s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 6m 45s | |
hadoop-hdfs-project-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 1134
unchanged - 1 fixed = 1134 total (was 1135) |
| +1 :green_heart: | compile | 6m 34s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 6m 34s | |
hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 1103
unchanged - 1 fixed = 1103 total (was 1104) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 26s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 52s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 16s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 41s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 251m 46s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not
generate ASF License warnings. |
| | | 408m 50s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4660/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4660 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux aef4a00016af 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 07dd3584cd7543a033a6c510
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17579904#comment-17579904 ] ASF GitHub Bot commented on HDFS-16703: --- goiri commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1215773135 > @slfan1989 Thanks for your review. > > > Do we need to update the md file to explain what these parameters do? > > Do you mean which MD file? `ipc.client.rpc-timeout.ms` is not been explained. If you think we should do it, I will add some explanation for them. I'm not sure what is the best place to document. TBH, I do not see a good place to document this; so it might be best to just have it in the XML. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17580834#comment-17580834 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1218141621 @slfan1989 Master, ping. Could you review it and make some suggestions or approval? > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583246#comment-17583246 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1223313799 @slfan1989 Master, ping. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583270#comment-17583270 ] ASF GitHub Bot commented on HDFS-16703: --- slfan1989 commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1223374325 @ZanderXu Sorry for the late reply, the code looks fine! Thank you very much for your contribution! > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583271#comment-17583271 ] ASF GitHub Bot commented on HDFS-16703: --- slfan1989 commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1223375293 I'm really worried that these timeout configurations are too much for the user, because from my personal point of view, I don't know how to configure it, or if there is some experience that can provide. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583293#comment-17583293 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1223433624 @slfan1989 Master, thanks for your review. > ipc.rpc-timeout.for.refresh-user-mappings.ms, if this is not configured to 0, what is a better configuration value? I have added some suggestion values in hdfs-default.xml, such as: ``` The default value of 0 indicates that timeout is disabled, which can be set to the same as ipc.client.rpc-timeout.ms, such as 120s. ``` What do you think of this? > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[
https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583511#comment-17583511
]
ASF GitHub Bot commented on HDFS-16703:
---
hadoop-yetus commented on PR #4660:
URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1223879828
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 41s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 17s | | trunk passed |
| +1 :green_heart: | compile | 6m 59s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 6m 21s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 49s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 12s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 19s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 19s | | the patch passed |
| +1 :green_heart: | compile | 6m 37s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 6m 37s | |
hadoop-hdfs-project-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 1134
unchanged - 1 fixed = 1134 total (was 1135) |
| +1 :green_heart: | compile | 6m 22s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 6m 22s | |
hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with
JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 1103
unchanged - 1 fixed = 1103 total (was 1104) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 41s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 7s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 29s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 349m 13s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4660/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not
generate ASF License warnings. |
| | | 506m 47s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.namenode.ha.TestObserverNode |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4660/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4660 |
| Optional Tests | dupname asflicense compile javac javadoc mv
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598313#comment-17598313 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1232760518 @slfan1989 Sir, can help take a look about this PR? It's helpful for hadoop admin. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600560#comment-17600560 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1237612512 @slfan1989 Sir, sorry to ping you again. I still think this is a useful PR for Hadoop Admin. So I hope you can give me some more suggestions and push this PR forward. Thanks again. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[
https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600698#comment-17600698
]
ASF GitHub Bot commented on HDFS-16703:
---
hadoop-yetus commented on PR #4660:
URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1237912143
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 46s | | trunk passed |
| +1 :green_heart: | compile | 6m 6s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 5m 55s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 37s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 14s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 36s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 4s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 14s | | the patch passed |
| +1 :green_heart: | compile | 6m 1s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 6m 1s | |
hadoop-hdfs-project-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 0 new + 1134 unchanged - 1
fixed = 1134 total (was 1135) |
| +1 :green_heart: | compile | 5m 41s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 5m 41s | |
hadoop-hdfs-project-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 with
JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 generated 0 new + 1103
unchanged - 1 fixed = 1103 total (was 1104) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 8s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 37s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 38s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 239m 8s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 8s | | The patch does not
generate ASF License warnings. |
| | | 385m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4660/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4660 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux ae65e76df820 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 9967ad85b45410dd431391942dd7faad1eedbad0 |
| Default Jav
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17605271#comment-17605271 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1247953589 @slfan1989 Sir, sorry to ping you again. I'm looking forward to your feedback. RPC Timeout is needed to prevent long-term blocking when the network is bad or machine crash. Thanks again. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17607473#comment-17607473 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1253151296 @Hexiaoqiao @ayushtkn Sir, can you help me review this patch? Enable RPC timeout for some protocol. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617449#comment-17617449 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1278453491 @slfan1989 Sir, about this PR, if you thinks it's difficult to configure it, how about just enabling a configurable timeout for NamenodeProtocolPB? Because we encountered many times this problem in our prod environment that RBF can not sense the crashed namenode in time, because the `NamenodeHeartbeatService` are blocked to waiting for the response from namenode for a long time. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
