[
https://issues.apache.org/jira/browse/FLINK-20682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17252719#comment-17252719
]
Till Rohrmann commented on FLINK-20682:
---------------------------------------
Thanks for creating this issue [~jiang7chengzitc]. I can see the benefits of
it. Would you be willing to implement it?
> Add configuration options related to hadoop
> -------------------------------------------
>
> Key: FLINK-20682
> URL: https://issues.apache.org/jira/browse/FLINK-20682
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Affects Versions: 1.12.0
> Reporter: Ruguo Yu
> Priority: Major
> Labels: pull-requests-available
> Fix For: 1.13.0, 1.12.1
>
>
> Current, we submit flink job to yarn with run-application target and need to
> specify some configuration related to hadoop, because we use distributed
> filesystem similar to Ali oss to storage resources, in this case, we will
> pass special configuration option and set them to hadoopConfiguration.
> In order to solve such problems, we can provide a configuration option
> prefixed with "flink.hadoop."(such as -Dflink.hadoop.xxx=yyy), and then take
> it into HadoopConfiguration.
> A simple implementation code is as follows:
> {code:java}
> module: flink-filesystems/flink-hadoop-fs
> class: org.apache.flink.runtime.util.HadoopUtils
> //代码占位符
> public static Configuration
> getHadoopConfiguration(org.apache.flink.configuration.Configuration
> flinkConfiguration) {
> ......
> // Copy any "flink.hadoop.xxx=yyy" flink configuration to hadoop
> configuration as "xxx=yyy"
> for (String key : flinkConfiguration.keySet()) {
> if (key.startsWith("flink.hadoop.")) {
> result.set(key.substring("flink.hadoop.".length()),
> flinkConfiguration.getString(key, null));
> }
> }
> return result;
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
