[
https://issues.apache.org/jira/browse/HDFS-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386384#comment-16386384
]
Íñigo Goiri commented on HDFS-13214:
------------------------------------
Thanks [~linyiqun] for catching the typo in {{namenodeHeartbeatServices}}.
The unit tests look good.
Minor comments on the doc:
{code}
Note: The config *dfs.nameservice.id* is recommended to configure if
*dfs.federation.router.monitor.localnamenode.enable* is enabled.
This will allow the Router finding the local node directly. Otherwise, it will
find the nameservice Id by matching namenode RPC address with the
local node address. If multiple addresses are matched, the Router will fail to
start. In addition, if the local node is in a HA mode, it is recommend
to configure *dfs.ha.namenode.id*.
{code}
bq. It's a good point. One way I am thinking that we can construct some
simulated NN/DNs and don't start real nodes/clusters. If there are some test
cases which need many namespaces, this will spend a lot of time to start.
We have some unit tests in RBF that don't rely on the whole MiniDFSCluster,
it'd be to have more of those.
Maybe we should open a JIRA for that; on March 12th there is a bug bash for the
build system, we may target this there too.
[~Tao Jie], does this tackle your issues?
> RBF: Configuration on Router conflicts with client side configuration
> ---------------------------------------------------------------------
>
> Key: HDFS-13214
> URL: https://issues.apache.org/jira/browse/HDFS-13214
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 2.9.0
> Reporter: Tao Jie
> Assignee: Yiqun Lin
> Priority: Major
> Attachments: HDFS-13214.001.patch, HDFS-13214.002.patch,
> HDFS-13214.003.patch
>
>
> In a typical router-based federation cluster, hdfs-site.xml is supposed to be:
> {code}
> <property>
> <name>dfs.nameservices</name>
> <value>ns1,ns2,ns-fed</value>
> </property>
> <property>
> <name>dfs.ha.namenodes.ns-fed</name>
> <value>r1,r2</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.ns1</name>
> <value>host1:8020</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.ns2</name>
> <value>host2:8020</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.ns-fed.r1</name>
> <value>host1:8888</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.ns-fed.r2</name>
> <value>host2:8888</value>
> </property>
> {code}
> {{dfs.ha.namenodes.ns-fed}} here is used for client to access the Router.
> However with this configuration on server node, Router fails to start with
> error:
> {code}
> org.apache.hadoop.HadoopIllegalArgumentException: Configuration has multiple
> addresses that match local node's address. Please configure the system with
> dfs.nameservice.id and dfs.ha.namenode.id
> at org.apache.hadoop.hdfs.DFSUtil.getSuffixIDs(DFSUtil.java:1198)
> at org.apache.hadoop.hdfs.DFSUtil.getNameServiceId(DFSUtil.java:1131)
> at
> org.apache.hadoop.hdfs.DFSUtil.getNamenodeNameServiceId(DFSUtil.java:1086)
> at
> org.apache.hadoop.hdfs.server.federation.router.Router.createLocalNamenodeHearbeatService(Router.java:466)
> at
> org.apache.hadoop.hdfs.server.federation.router.Router.createNamenodeHearbeatServices(Router.java:423)
> at
> org.apache.hadoop.hdfs.server.federation.router.Router.serviceInit(Router.java:199)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.hdfs.server.federation.router.DFSRouter.main(DFSRouter.java:69)
> 2018-03-01 18:05:56,208 ERROR
> org.apache.hadoop.hdfs.server.federation.router.DFSRouter: Failed to start
> router
> {code}
> Then the router tries to find the local namenode, multiple properties:
> {{dfs.namenode.rpc-address.ns1}}, {{dfs.namenode.rpc-address.ns-fed.r1}}
> match the local address.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
