[
https://issues.apache.org/jira/browse/HDFS-15905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304444#comment-17304444
]
Íñigo Goiri commented on HDFS-15905:
------------------------------------
If I understand correctly, the proposal is to extend the client to query the
Router and then contact the subcluster directly?
To be honest, this is very similar to ViewFs; you could potentially extend
ViewFs to request the mount table from the Router.
I'll let other chime in on the token aspect.
Regarding the performance issues, how many Routers are you using for how many
namenodes?
> RBF: Improve Router performance with router redirection
> -------------------------------------------------------
>
> Key: HDFS-15905
> URL: https://issues.apache.org/jira/browse/HDFS-15905
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: rbf
> Affects Versions: 3.1.0
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
>
> Router implementation currently takes the proxy approach to handle the client
> requests: the routers receive the requests from the clients and send the
> requests to the target clusters on behalf of the clients.
> This approach works well, while after moving more clusters on top of
> routers, we are seeing that routers are becoming the bottleneck since e.g.,
> without RBF, the clients themselves manage the connections for themselves,
> while with RBF, the limited routers manage much more connections for the
> clients; we also keep idle connections to boost the connection performance.
> We have done some work to tune connection management but it doesn't help much.
> We are proposing to reduce the functionality on the router side and use them
> as actual router instead of proxy: the clients talk to routers to resolve
> target cluster info given a path and get router delegation token; the clients
> directly send the requests to target cluster.
> A big challenge here is the token authentication against target cluster with
> router token only. One approach: we can ask router to return target cluster
> token along with router token so the clients can authenticate against target
> cluster. Second approach: similar to block token mechanism, the router
> exchanges secret keys with target clusters through heart-beats so the clients
> can authenticate with target cluster with that router token.
> I would like to know your feedback.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
