[
https://issues.apache.org/jira/browse/HDFS-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786382#comment-16786382
]
He Xiaoqiao commented on HDFS-13532:
------------------------------------
[~crh],[~elgoiri],[~brahmareddy], really appreciate your feedback.
Basically I am concerning 2 points in one word:
* (1) how to gray upgrade HDFS to support RBF with security feature.
* (2) performance cost using ZKDelegationTokenSecretManagerImpl.
And It is clear about (2) performance of ZKDelegationTokenSecretManagerImpl
with my colleague's help. it is OK for me that >5K QPS.
I do not understand about gray upgrade completely. First of all, I would like
to share ideal plan for me to upgrade RBF smoothly: (1) HDFS build on
Federation + ViewFS now. (2) It's better for me to rolling upgrade Client
rather than switch to RBF once time.
[~elgoiri] and [~crh] both mentioned solution with 'Router nameservice' as
following step:
* (1) update YARN(RM/NM) configuration within new router nameservice;
* (2) rolling client to support RBF;
* (3) updete YARN(RM/NM) configuration which include router nameservice config
only;
IIUC, this solution will not solve delegation token issue, since client obtains
DT from router only after step (2) and submit job normally, however executor
will fail when request to NameNode due to DT checks fail, since for some
compute engine (for instance MR) it merges client and NM configuration
together, then executor still request to NameNode directly without proper DT.
To [~crh]
{quote}jobs try to access something like hdfs://router-nameservice/mydata, rm
will use the same filesystem i.e. hdfs://router-nameservice to renew tokens
{quote}
I think it need to enhance compute engine, may be more high-cost.
{quote}Routers not having security feature was a big hindrance in adopting it
for any secure use case irrespective of scale.
{quote}
security feature is also very important for me, I try my best to dig solution
that can transmit to RBF smoothly.
Thanks [~crh], [~elgoiri] again.
> RBF: Adding security
> --------------------
>
> Key: HDFS-13532
> URL: https://issues.apache.org/jira/browse/HDFS-13532
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Íñigo Goiri
> Assignee: CR Hota
> Priority: Major
> Attachments: RBF _ Security delegation token thoughts.pdf, RBF _
> Security delegation token thoughts_updated.pdf, RBF _ Security delegation
> token thoughts_updated_2.pdf, RBF-DelegationToken-Approach1b.pdf, RBF_
> Security delegation token thoughts_updated_3.pdf, Security_for_Router-based
> Federation_design_doc.pdf
>
>
> HDFS Router based federation should support security. This includes
> authentication and delegation tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
