Thanks for kicking off the discussion. I think supporting environment variables rendering in the Flink configuration yaml file is a good idea. Especially for the Kubernetes environment since we are using the secret resource to store the authentication information.
But I have some questions for how to do it? 1. The environments in Flink configuration yaml will be replaced in client, JobManager, TaskManager or all of them? 2. If users do not want some config options to be replaced, how to achieve that? Best, Yang Khachatryan Roman <khachatryan.ro...@gmail.com> 于2021年1月18日周一 下午8:55写道： > Hi Ingo, > > Thanks a lot for this proposal! > > We had a related discussion recently in the context of FLINK-19520 > (randomizing tests configuration) [1]. > I believe other scenarios will benefit as well. > > For the end users, I think substitution in configuration files is > preferable over parsing env vars in Flink code. > And for cases without such a file, we could have a default one on the > classpath with all substitutions defined (and then merge everything from > the user-supplied file). > > [1] https://issues.apache.org/jira/browse/FLINK-19520 > > Regards, > Roman > > > On Mon, Jan 18, 2021 at 11:11 AM Ingo Bürk <i...@ververica.com> wrote: > > > Hi everyone, > > > > in Ververica Platform we offer a feature to use environment variables in > > the Flink configuration¹, e.g. > > > > ``` > > s3.access-key: ${S3_ACCESS_KEY} > > ``` > > > > We've been discussing internally whether contributing such a feature to > > Flink directly would make sense and wanted to start a discussion on this > > topic. > > > > An alternative way to do so from the above would be parsing those > directly > > based on their name, so instead of having it defined in the Flink > > configuration as above, it would get automatically set if something like > > $FLINK_CONFIG_S3_ACCESS_KEY was set in the environment. This is somewhat > > similar to what e.g. Spring does, and faces similar challenges (dealing > > with "."s etc.) > > > > Although I view both of these approaches as mostly orthogonal, supporting > > both very likely wouldn't make sense, of course. So I was wondering what > > your opinion is in terms of whether the project would benefit from > > environment variable support for the Flink configuration, and whether > there > > are tendencies as to which approach to go with. > > > > ¹ > > > > > https://docs.ververica.com/user_guide/application_operations/deployments/configure_flink.html#environment-variables > > > > Best regards > > Ingo > > >
