[
https://issues.apache.org/jira/browse/HDFS-15905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated HDFS-15905:
-------------------------------
Summary: RBF: Improve Router performance with router redirection (was:
Improve Router performance with router redirection)
> RBF: Improve Router performance with router redirection
> -------------------------------------------------------
>
> Key: HDFS-15905
> URL: https://issues.apache.org/jira/browse/HDFS-15905
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: rbf
> Affects Versions: 3.1.0
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
>
> Router implementation currently takes the proxy approach to handle the client
> requests: the routers receive the requests from the clients and send the
> requests to the target clusters on behalf of the clients.
> This approach works well, while after moving more clusters on top of
> routers, we are seeing that routers are becoming the bottleneck since e.g.,
> without RBF, the clients themselves manage the connections for themselves,
> while with RBF, the limited routers manage much more connections for the
> clients; we also keep idle connections to boost the connection performance.
> We have done some work to tune connection management but it doesn't help much.
> We are proposing to reduce the functionality on the router side and use them
> as actual router instead of proxy: the clients talk to routers to resolve
> target cluster info given a path and get router delegation token; the clients
> directly send the requests to target cluster.
> A big challenge here is the token authentication against target cluster with
> router token only. One approach: we can ask router to return target cluster
> token along with router token so the clients can authenticate against target
> cluster. Second approach: similar to block token mechanism, the router
> exchanges secret keys with target clusters through heart-beats so the clients
> can authenticate with target cluster with that router token.
> I would like to know your feedback.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
