[
https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382547#comment-15382547
]
Inigo Goiri commented on HDFS-10467:
------------------------------------
Thanks [~vinayrpet] for the comments.
bq. Why cant we use DFSClient with HA here directly? Like how current clients
connects to HA, each subcluster can be connected to a subcluster using a HA
configured DFSClient. DFSClient itself will handle switching between NNs in
case of failover. This DFSClient can be kept as-is and re-used later for next
requests on same subcluster. So it should know the current active namenode.
To provide a fully federated view, we think is best to track the state of all
the Namenodes. In this way, we can expose the federation view in the web UI.
Given that we have this information, we can use this information as hints for
the clients. Actually, there was some discussion in HDFS-7858 regarding using
the information in ZK to go faster to the Active namenode. This was discarded
because of the additional complexity. I think this might be a good opportunity
to go in that direction. Our current implementation (using the Active hint) is
faster than the regular fail over and produces less load than the hedging
approach.
bq. How about supporting a default Mount-point for '/' as well. Could be
optional also? Instead of rejecting requests for the paths which doesnt match
with other other mountpoints.
There might be some usecases where there might be multiple first level
directories other than mounted path. Which could go under /.
Yes, this is a common use case. We already support a default / set using
{{dfs.router.default.nameserviceId}}. We may want to make it more
explicit/clear.
> Router-based HDFS federation
> ----------------------------
>
> Key: HDFS-10467
> URL: https://issues.apache.org/jira/browse/HDFS-10467
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: fs
> Affects Versions: 2.7.2
> Reporter: Inigo Goiri
> Assignee: Inigo Goiri
> Attachments: HDFS Router Federation.pdf, HDFS-10467.PoC.001.patch,
> HDFS-10467.PoC.patch, HDFS-Router-Federation-Prototype.patch
>
>
> Add a Router to provide a federated view of multiple HDFS clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
