[
https://issues.apache.org/jira/browse/FLINK-14360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954669#comment-16954669
]
Andrey Zagrebin commented on FLINK-14360:
-----------------------------------------
ok, this other type of federation does not handle the tokens if I understand
correctly.
What it is not quite clear why tokens are not obtained in the current
implementation. Flink needs them only for certain files used for deployment and
their tokens are specifically asked for each file so I would expect that the
token requests would be routed and returned from the necessary clusters for
each file. Or I miss something here?
Spark documentation says that hadoopFileSystems should be set if application
needs to access multiple clusters. User applications in Flink access hdfs over
the FileSystem abstraction and PR modifies the deployment part which is Flink
specific, not user application. So is this change about user application hdfs
access or for Flink on Yarn deployment?
> Flink on yarn should support obtain delegation tokens of multi hdfs namespaces
> ------------------------------------------------------------------------------
>
> Key: FLINK-14360
> URL: https://issues.apache.org/jira/browse/FLINK-14360
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Reporter: Shen Yinjie
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There's a scenario when deploy flink on yarn with multi hdfs cluster or hdfs
> federation, Flink need to get delegation tokens of all the namespaces before
> start appmaster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
