[jira] [Updated] (HDFS-8820) Simplify enabling NameNode RPC congestion control and FairCallQueue
[ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8820: - Target Version/s: (was: 2.8.0) > Simplify enabling NameNode RPC congestion control and FairCallQueue > --- > > Key: HDFS-8820 > URL: https://issues.apache.org/jira/browse/HDFS-8820 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch, > HDFS-8820.03.patch > > > Enabling RPC Congestion control and FairCallQueue settings can be simplified > with HDFS-specific configuration keys. Currently the configuration requires > knowing the exact RPC port number and also whether the service RPC port is > enabled or not separately. If a separate service RPC endpoint is not defined > then RPC congestion control must be enabled ([see > comment|https://issues.apache.org/jira/browse/HDFS-8820?focusedCommentId=14987848&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14987848] > from [~mingma] below. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-8820) Simplify enabling NameNode RPC congestion control and FairCallQueue
[ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8820: Status: Open (was: Patch Available) > Simplify enabling NameNode RPC congestion control and FairCallQueue > --- > > Key: HDFS-8820 > URL: https://issues.apache.org/jira/browse/HDFS-8820 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch, > HDFS-8820.03.patch > > > Enabling RPC Congestion control and FairCallQueue settings can be simplified > with HDFS-specific configuration keys. Currently the configuration requires > knowing the exact RPC port number and also whether the service RPC port is > enabled or not separately. If a separate service RPC endpoint is not defined > then RPC congestion control must be enabled ([see > comment|https://issues.apache.org/jira/browse/HDFS-8820?focusedCommentId=14987848&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14987848] > from [~mingma] below. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8820) Simplify enabling NameNode RPC congestion control and FairCallQueue
[ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8820: Attachment: (was: HDFS-8820.03.patch) > Simplify enabling NameNode RPC congestion control and FairCallQueue > --- > > Key: HDFS-8820 > URL: https://issues.apache.org/jira/browse/HDFS-8820 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch, > HDFS-8820.03.patch > > > Enabling RPC Congestion control and FairCallQueue settings can be simplified > with HDFS-specific configuration keys. Currently the configuration requires > knowing the exact RPC port number and also whether the service RPC port is > enabled or not separately. If a separate service RPC endpoint is not defined > then RPC congestion control must be enabled ([see > comment|https://issues.apache.org/jira/browse/HDFS-8820?focusedCommentId=14987848&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14987848] > from [~mingma] below. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8820) Simplify enabling NameNode RPC congestion control and FairCallQueue
[ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8820: Attachment: HDFS-8820.03.patch > Simplify enabling NameNode RPC congestion control and FairCallQueue > --- > > Key: HDFS-8820 > URL: https://issues.apache.org/jira/browse/HDFS-8820 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch, > HDFS-8820.03.patch > > > Enabling RPC Congestion control and FairCallQueue settings can be simplified > with HDFS-specific configuration keys. Currently the configuration requires > knowing the exact RPC port number and also whether the service RPC port is > enabled or not separately. If a separate service RPC endpoint is not defined > then RPC congestion control must be enabled ([see > comment|https://issues.apache.org/jira/browse/HDFS-8820?focusedCommentId=14987848&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14987848] > from [~mingma] below. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8820) Simplify enabling NameNode RPC congestion control and FairCallQueue
[ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8820: Attachment: HDFS-8820.03.patch Changed the title and description to reflect the changed intent. We got requests to make these features easier to configure and also include the service RPC port check within HDFS so if the administrator does the wrong thing we don't silently break DN-NN communication at runtime (same thing mentioned by [~mingma] above). v03 patch allows enabling congestion control and fair call queue via NameNode configuration settings and includes service RPC port check. I still communicate intent from NameNodeRpcServer to IPC Server via config settings since I am unsure whether changing the RPCEngine interface will be compatible. It is somewhat ugly so if someone can confirm the RPCEngine interface can be changed I can use the builder approach suggested by Ming. > Simplify enabling NameNode RPC congestion control and FairCallQueue > --- > > Key: HDFS-8820 > URL: https://issues.apache.org/jira/browse/HDFS-8820 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch, > HDFS-8820.03.patch > > > Enabling RPC Congestion control and FairCallQueue settings can be simplified > with HDFS-specific configuration keys. Currently the configuration requires > knowing the exact RPC port number and also whether the service RPC port is > enabled or not separately. If a separate service RPC endpoint is not defined > then RPC congestion control must be enabled ([see > comment|https://issues.apache.org/jira/browse/HDFS-8820?focusedCommentId=14987848&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14987848] > from [~mingma] below. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8820) Simplify enabling NameNode RPC congestion control and FairCallQueue
[ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8820: Description: Enabling RPC Congestion control and FairCallQueue settings can be simplified with HDFS-specific configuration keys. Currently the configuration requires knowing the exact RPC port number and also whether the service RPC port is enabled or not separately. If a separate service RPC endpoint is not defined then RPC congestion control must be enabled ([see comment|https://issues.apache.org/jira/browse/HDFS-8820?focusedCommentId=14987848&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14987848] from [~mingma] below. (was: Enabling RPC Congestion control and FairCallQueue settings can be simplified with HDFS-specific configuration keys. Currently the configuration requires knowing the exact RPC port number and also whether the service RPC port is enabled or not separately. If a separate service RPC endpoint is not defined then RPC congestion control must be enabled. ) > Simplify enabling NameNode RPC congestion control and FairCallQueue > --- > > Key: HDFS-8820 > URL: https://issues.apache.org/jira/browse/HDFS-8820 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch > > > Enabling RPC Congestion control and FairCallQueue settings can be simplified > with HDFS-specific configuration keys. Currently the configuration requires > knowing the exact RPC port number and also whether the service RPC port is > enabled or not separately. If a separate service RPC endpoint is not defined > then RPC congestion control must be enabled ([see > comment|https://issues.apache.org/jira/browse/HDFS-8820?focusedCommentId=14987848&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14987848] > from [~mingma] below. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8820) Simplify enabling NameNode RPC congestion control and FairCallQueue
[ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8820: Description: Enabling RPC Congestion control and FairCallQueue settings can be simplified with HDFS-specific configuration keys. Currently the configuration requires knowing the exact RPC port number and also whether the service RPC port is enabled or not separately. If a separate service RPC endpoint is not defined then RPC congestion control must be enabled. (was: We propose enabling RPC congestion control introduced by HADOOP-10597 by default. We enabled it on a couple of large clusters a few weeks ago and it has helped keep the namenodes responsive under load.) > Simplify enabling NameNode RPC congestion control and FairCallQueue > --- > > Key: HDFS-8820 > URL: https://issues.apache.org/jira/browse/HDFS-8820 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch > > > Enabling RPC Congestion control and FairCallQueue settings can be simplified > with HDFS-specific configuration keys. Currently the configuration requires > knowing the exact RPC port number and also whether the service RPC port is > enabled or not separately. If a separate service RPC endpoint is not defined > then RPC congestion control must be enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8820) Simplify enabling NameNode RPC congestion control and FairCallQueue
[ https://issues.apache.org/jira/browse/HDFS-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8820: Summary: Simplify enabling NameNode RPC congestion control and FairCallQueue (was: Enable RPC Congestion control by default) > Simplify enabling NameNode RPC congestion control and FairCallQueue > --- > > Key: HDFS-8820 > URL: https://issues.apache.org/jira/browse/HDFS-8820 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal > Attachments: HDFS-8820.01.patch, HDFS-8820.02.patch > > > We propose enabling RPC congestion control introduced by HADOOP-10597 by > default. > We enabled it on a couple of large clusters a few weeks ago and it has helped > keep the namenodes responsive under load. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
