[
https://issues.apache.org/jira/browse/FLINK-14360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949883#comment-16949883
]
Shen Yinjie commented on FLINK-14360:
-------------------------------------
Thanks [~azagrebin] for comments!
1)Current "paths" only contains some paths as
${fs.defaultFS}/user/{user.name}/.flink/appID/xxx according to client's
fs.defaultFS setting.
2)For example, when we deploy flink on a Hadoop cluster with YARN
federation(two YARN subclusters have separate HDFSs, one HDFS enable RBF and
another enable HA). Flink should access RBF's ns1, ns2 and router-fed ns,
sometimes another hdfs will be accessed.Since YARN router will forward
client's flink job to a random YARN subcluster by some polices.
3)So, we add a new config in flink-conf.yaml for users to set neccessary HDFS
schemes to obtain tokens.
Not sure am I clear :)
> Flink on yarn should support obtain delegation tokens of multi hdfs namespaces
> ------------------------------------------------------------------------------
>
> Key: FLINK-14360
> URL: https://issues.apache.org/jira/browse/FLINK-14360
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Reporter: Shen Yinjie
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There's a scenario when deploy flink on yarn with multi hdfs cluster or hdfs
> federation, Flink need to get delegation tokens of all the namespaces before
> start appmaster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
