Re: ClusterClientFactory selection

2020-05-28 Thread Yang Wang
You could find more information about deployment target here[1]. As you mentioned, it is not defined in the flink-conf.yaml by default. For the code, it is defined in flink-core/DeploymentOptions. [1]. https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html#deployment-targets Best,

Re: ClusterClientFactory selection

2020-05-28 Thread M Singh
HI Kostas/Yang/Lake: I am looking at aws emr and did not see the execution.target in the flink-conf.yaml file under flink/conf directory. Is it defined in another place ?   I also did search in the current flink source code and did find mention of it in the md files but not in any property

Re: ClusterClientFactory selection

2020-05-27 Thread Kostas Kloudas
Hi Singh, The only thing to add to what Yang said is that the "execution.target" configuration option (in the config file) is also used for the same purpose from the execution environments. Cheers, Kostas On Wed, May 27, 2020 at 4:49 AM Yang Wang wrote: > > Hi M Singh, > > The Flink CLI picks

Re: ClusterClientFactory selection

2020-05-26 Thread Yang Wang
Hi M Singh, The Flink CLI picks up the correct ClusterClientFactory via java SPI. You could check YarnClusterClientFactory#isCompatibleWith for how it is activated. The cli option / configuration is "-e/--executor" or execution.target (e.g. yarn-per-job)*.* Best, Yang M Singh 于2020年5月26日周二

ClusterClientFactory selection

2020-05-26 Thread M Singh
Hi: I wanted to find out which parameter/configuration allows flink cli pick up the appropriate cluster client factory (especially in the yarn mode). Thanks