[
https://issues.apache.org/jira/browse/FLINK-16305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046473#comment-17046473
]
Gyula Fora commented on FLINK-16305:
------------------------------------
[~kkl0u], I don't think this is question of backwards compatibility. If I
understand it correctly the execution-target configuration was added after the
property magic.
The current behaviour is simply broken:
1. You set execution target to yarn-per-job
2. FlinkYarnSessionCLI gets activated as it sees the execution target
3. FlinkYarnSessionCLI completely ignores the target and overwrites to session
This is broken and is very easy to fix in the FlinkYarnSessionCLI itself.
> FlinkYarnSessionClI ignores target executor and uses yarn-session if YARN
> properties file is present
> ----------------------------------------------------------------------------------------------------
>
> Key: FLINK-16305
> URL: https://issues.apache.org/jira/browse/FLINK-16305
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission, Command Line Client
> Affects Versions: 1.10.0
> Reporter: Daniel Laszlo Magyar
> Priority: Major
>
> The presence of the hidden YARN property file (which contains the last
> started YARN session’s application id), causes the cli to ignore the
> {{execution.target}} property set in the {{conf/flink-conf.yaml}}
> configuration file, which leads to unexpected behaviour at the time of job
> submission via cli, e.g. when using {{flink run}} or SQL client.
> The code that ignores the execution target if the YARN application id is set
> in the hidden property file is at
> [https://github.com/apache/flink/blob/release-1.10/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L337-L351].
>
> Reproduction steps:
> • start flink yarn session via {{./bin/yarn-session.sh -d}}, this writes the
> application id to {{/tmp/.yarn-properties-root}}
> • set {{execution.target: yarn-per-job}} in
> {{/etc/flink/conf/flink-conf.yaml}}
> • enable debug logging
> • run a flink job e.g. {{flink run -d -p 2 examples/streaming/WordCount.jar
> --input README.txt}}
> • the logs below show that even though the {{execution.target}} property is
> read properly, {{FlinkYarnSessionCli}} is chosen and the execution.target is
> reset to yarn-session
> {code:java}
> 20/02/26 12:14:24 INFO configuration.GlobalConfiguration: Loading
> configuration property: execution.target, yarn-per-job
> ...
> 20/02/26 12:14:24 INFO cli.FlinkYarnSessionCli: Found Yarn properties file
> under /tmp/.yarn-properties-root.
> 20/02/26 12:14:24 DEBUG fs.FileSystem: Loading extension file systems via
> services
> 20/02/26 12:14:24 DEBUG cli.CliFrontend: Custom commandlines:
> [org.apache.flink.yarn.cli.FlinkYarnSessionCli@43df23d3,
> org.apache.flink.client.cli.ExecutorCLI@6d60fe40,
> org.apache.flink.client.cli.DefaultCLI@792b749c]
> 20/02/26 12:14:24 DEBUG cli.CliFrontend: Checking custom commandline
> org.apache.flink.yarn.cli.FlinkYarnSessionCli@43df23d3, isActive: true
> ...
> 20/02/26 12:14:25 DEBUG cli.CliFrontend: Effective executor configuration:
> {...execution.target=yarn-session, ....}
> 20/02/26 12:14:25 INFO client.ClientUtils: Starting program (detached: true)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
