[jira] [Commented] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339602#comment-17339602
]
Dawid Wysakowicz commented on FLINK-21153:
--
No, not actively. I think we need to agree on what is to be done with this
ticket. I will unassign myself.
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.1, 1.13.0
>Reporter: Till Rohrmann
>Assignee: Dawid Wysakowicz
>Priority: Major
> Labels: stale-assigned, usability
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339596#comment-17339596
]
Till Rohrmann commented on FLINK-21153:
---
[~dwysakowicz] are you still actively looking into this ticket?
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.1, 1.13.0
>Reporter: Till Rohrmann
>Assignee: Dawid Wysakowicz
>Priority: Major
> Labels: stale-assigned, usability
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322855#comment-17322855
]
Flink Jira Bot commented on FLINK-21153:
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.1, 1.13.0
>Reporter: Till Rohrmann
>Assignee: Dawid Wysakowicz
>Priority: Major
> Labels: stale-assigned, usability
> Fix For: 1.13.0, 1.12.3
>
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17292980#comment-17292980
]
Dawid Wysakowicz commented on FLINK-21153:
--
I think the answer is that the intention was to use only the config options and
get rid of the {{Options}}. In order to do that if I understand it correctly
the plan was to add support for printing help messages/handling of custom
options to the {{ClusterClientFactory}}. I am afraid this has not been
implemented yet.
As for how you can ship files with the {{-t}} option I think you can do it like
this:
{code}
bin/flink run -p 1
-Dyarn.ship-files=../flink-test-job/cluster;../flink-test-job/cluster2 -t
yarn-per-job ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar
{code}
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.1, 1.13.0
>Reporter: Till Rohrmann
>Assignee: Dawid Wysakowicz
>Priority: Major
> Labels: usability
> Fix For: 1.13.0, 1.12.3
>
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17291542#comment-17291542
]
Chesnay Schepler commented on FLINK-21153:
--
[~dwysakowicz] ping
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.1, 1.13.0
>Reporter: Till Rohrmann
>Priority: Major
> Labels: usability
> Fix For: 1.13.0, 1.12.3
>
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17272858#comment-17272858
]
Till Rohrmann commented on FLINK-21153:
---
How are we implementing features which only work for a special deployment
target and wouldn't work in the generic case (e.g. shipping files to the
cluster)?
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.13.0, 1.12.1
>Reporter: Till Rohrmann
>Priority: Major
> Labels: usability
> Fix For: 1.13.0, 1.12.2
>
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17272731#comment-17272731
]
Aljoscha Krettek commented on FLINK-21153:
--
I agree with [~fly_in_gis], the behaviour of the {{GenericCLI}} is different
(simpler) and eventually the special behavior we have for YARN should be
removed.
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.13.0, 1.12.1
>Reporter: Till Rohrmann
>Priority: Major
> Labels: usability
> Fix For: 1.13.0, 1.12.2
>
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17272541#comment-17272541
]
Yang Wang commented on FLINK-21153:
---
IIUC, this is the expected behavior. Because {{-t,--target}} will always active
the {{GenericCLI}}. Actually, {{FlinkYarnSessionCli}} should be deprecated and
removed in the future. But it seems it is widely used(from command line
submission) and we could not do it right now.
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.13.0, 1.12.1
>Reporter: Till Rohrmann
>Priority: Major
> Labels: usability
> Fix For: 1.13.0, 1.12.2
>
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
