[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673670#comment-16673670
]
Erik Krogen commented on HDFS-14017:
------------------------------------
So the way I see it, ObserverReadProxyProvider has two responsibilities: (A)
find the correct active, and (B) find the correct observer. CFPP-based ORPP
does both (A) and (B) by scanning through all NNs. IPFailoverPP, however, does
(A) without any scanning--we should maintain this behavior. Only for (B) should
ORPPWithIPFailover scan, because we don't have a good way to discover the
observers without scanning (we would need to introduce additional VIPs for
observers, and it's not clear to me if this makes sense). Reducing
ORPPWithIPFailover's ability to do (A) without scanning reduces its usefulness,
and essentially just introduces a way to have a CFPP whose configuration looks
like an IPFailoverPP.
{quote}Despite it is becoming very similar to ObserverReadProxyProvider, I
think there is still a very important difference, which is performFailover
method. The failoverproxy variable decides when everything failed, what to do.
For ConfiguredFailover, it automatically tries other NNs, while in IPFailover,
it should not change. (still missing this change in the patch though).
{quote}
If IPFailover uses the NN addresses directly rather than the VIP, to perform
failover, it will have to use a new proxy in the same way CFPP does. Otherwise,
when the active changes, how will it ever start using the new active?
{quote}
The configuration of parsing configured addresses is mainly to address the
delegation token. It needs to know all the addresses for delegation to work.
And I don't know if it is really hacky
{quote}
It was mainly the comment about not supporting federation that concerned me,
but looking more closely I think that is just a shortcoming of the current
patch because you didn't handle it yet, not a fundamental problem. So I'll take
back the hacky comment, I think I jumped to conclusions there.
{quote}So my take was that if we could already leverage existing configuration,
why do we need additional ones for the full set of NNs.
{quote}
Agree with this, I misunderstood the configuration parsing a bit, and it seems
that we can successfully get all of the NNs from the IPFailoverPP configs.
Great!
Overall, I think your methodology for fetching all of the NN addresses actually
makes sense, but then this should only be used for finding the observer /
cloning DTs, and let the IPFailoverPP continue to function as normal (using
VIP).
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -------------------------------------------------------------------------
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Chen Liang
> Assignee: Chen Liang
> Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
