[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16680534#comment-16680534
]
Erik Krogen commented on HDFS-14017:
------------------------------------
{quote}Probably should be based on different fs.defaultFS, which is what
IPFailover is largely relying on, maybe only a subset should be checked.
{quote}
I disagree that IPFPP is relying on {{fs.defaultFS}}. We clearly see from this
snippet that it loads the NN address of the provided URI, which may or may not
be the default. I don't see anywhere that IPFPP accesses the {{fs.defaultFS}}.
{code:java|title=IPFailoverProxyProvider}
public IPFailoverProxyProvider(Configuration conf, URI uri,
Class<T> xface, HAProxyFactory<T> factory) {
super(conf, uri, xface, factory);
this.nnProxyInfo = new NNProxyInfo<T>(DFSUtilClient.getNNAddress(uri));
}
{code}
{quote}In fact, in current patch, the call to DFSUtilClient.getAddresses(...)
is already getting ALL the host physical addresses for ALL the nameservices.
All these addresses will have proxies created and DT cloned. The issue I had
was that, it may not be true that all these addresses should be considered (and
clone DT for) in case of federation.
{quote}
Hm, I see your point now that, if you make the assumption that there is only
one nameservice considered, you can then assume that all of the configured
NameNodes are part of that nameservice. I still don't think that is the right
approach... We should be using
{{DFSUtilClient.getAddressesForNameserviceId()}}, which takes in a nameservice
ID, to return only NameNodes that are relevant to _this_ specific nameservice
ID. The nameservice ID can be determined by just {{nameNodeURI.getHost()}} –
the same convention followed by
{{NameNodeProxiesClient.getFailoverProxyProviderClass()}} to load the correct
proxy provider for the nameservice.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -------------------------------------------------------------------------
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Chen Liang
> Assignee: Chen Liang
> Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
